pgpool-II-2.3.3、PostgreSQL9.0.13の話題です。

pgpoolのオンラインリカバリをしたところ、recovery_2nd_stage.shの実行時に失敗しました。

recovery_2nd_stage.shの段階に入ると、pgpoolはクライアントからの要求を受け付けなくなります。
※recovery_2nd_stage.shはrecovery_1nd_stage.sh実行中に更新された内容をリカバリする側に反映する段階です。この段階で更新を受け付けてしまうと、その内容がバックアップ側に反映されないので、要求の受付を停止します。pgpoolのオンラインリカバリについてはpgpool-IIによるレプリケーションとオンラインリカバリを参照してください。

recovery_2nd_stage.shの段階で失敗すると被害が大きいです。
  • httpdは動いているのにpgpoolは要求を受け付けないのでサイトに接続できるけど応答に時間がかかったあげくDBに接続できなかった場合の画面が返る
  • pgpoolを止めるにはpgpoolのプロセスをすべてkill -HUPする必要がある
ので、サービスは止まる、pgpoolのプロセスは多くてkillするのが大変です。

この時はサーバーを再起動して復旧させました。
再起動後はちゃんとリカバリできましたので、設定誤りではなくて、何か違う要因なのですが、原因は分かりませんでした…

以下、その事象が発生した時の/var/log/messageになります。

Apr 13 12:16:27 erogamescape15 kernel: pgpool[7600]: segfault at 488 ip 000000000041866e sp 00007fffc4077d10 error 4 in pgpool[400000+bf000]
Apr 13 12:16:27 erogamescape15 kernel: pgpool[24018]: segfault at 488 ip 000000000041866e sp 00007fffc4077cd0 error 4 in pgpool[400000+bf000]
Apr 13 12:16:27 erogamescape15 kernel: pgpool[7629]: segfault at 488 ip 000000000041866e sp 00007fffc4077d10 error 4 in pgpool[400000+bf000]
Apr 13 12:16:27 erogamescape15 kernel: pgpool[6139]: segfault at 488 ip 000000000041866e sp 00007fffc4077cd0 error 4 in pgpool[400000+bf000]
Apr 13 12:16:27 erogamescape15 kernel: pgpool[7582]: segfault at 488 ip 000000000041866e sp 00007fffc4077d10 error 4 in pgpool[400000+bf000]
Apr 13 12:16:27 erogamescape15 kernel: Pid 7582(pgpool) over core_pipe_limit
Apr 13 12:16:27 erogamescape15 kernel: Skipping core dump
Apr 13 12:16:27 erogamescape15 kernel: pgpool[2703]: segfault at 488 ip 000000000041866e sp 00007fffc4077cd0 error 4 in pgpool[400000+bf000]
Apr 13 12:16:27 erogamescape15 kernel: Pid 2703(pgpool) over core_pipe_limit
Apr 13 12:16:27 erogamescape15 kernel: Skipping core dump
Apr 13 12:16:27 erogamescape15 kernel: pgpool[7669]: segfault at 488 ip 000000000041866e sp 00007fffc4077d10 error 4 in pgpool[400000+bf000]
Apr 13 12:16:27 erogamescape15 kernel: Pid 7669(pgpool) over core_pipe_limit
Apr 13 12:16:27 erogamescape15 kernel: Skipping core dump
Apr 13 12:16:27 erogamescape15 kernel: pgpool[7690]: segfault at 488 ip 000000000041866e sp 00007fffc4077d10 error 4 in pgpool[400000+bf000]
Apr 13 12:16:27 erogamescape15 kernel: Pid 7690(pgpool) over core_pipe_limit
Apr 13 12:16:27 erogamescape15 kernel: Skipping core dump
Apr 13 12:16:28 erogamescape15 abrt[6337]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:28 erogamescape15 kernel: pgpool[4253]: segfault at 488 ip 000000000041866e sp 00007fffc4077cd0 error 4 in pgpool[400000+bf000]
Apr 13 12:16:28 erogamescape15 abrt[6351]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:28 erogamescape15 abrt[6354]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:28 erogamescape15 abrt[6357]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:28 erogamescape15 abrt[6360]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:28 erogamescape15 abrt[6363]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:28 erogamescape15 abrt[6366]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:28 erogamescape15 abrt[6369]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:28 erogamescape15 abrt[6372]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:28 erogamescape15 abrt[6375]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:28 erogamescape15 abrt[6378]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:29 erogamescape15 abrt[6381]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:29 erogamescape15 abrt[6338]: Saved core dump of pid 24018 (/usr/local/bin/pgpool) to /var/spool/abrt/ccpp-2013-04-13-12:16:28-24018 (15818752 bytes)
Apr 13 12:16:29 erogamescape15 abrtd: Directory 'ccpp-2013-04-13-12:16:28-24018' creation detected
Apr 13 12:16:29 erogamescape15 abrt[6339]: Saved core dump of pid 7629 (/usr/local/bin/pgpool) to /var/spool/abrt/ccpp-2013-04-13-12:16:28-7629 (15826944 bytes)
Apr 13 12:16:29 erogamescape15 abrt[6340]: Saved core dump of pid 6139 (/usr/local/bin/pgpool) to /var/spool/abrt/ccpp-2013-04-13-12:16:28-6139 (15548416 bytes)
Apr 13 12:16:29 erogamescape15 abrt[6387]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:29 erogamescape15 abrt[6390]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:29 erogamescape15 abrt[6393]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:29 erogamescape15 abrt[6396]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:29 erogamescape15 abrt[6399]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:29 erogamescape15 abrt[6402]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:29 erogamescape15 abrt[6405]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:30 erogamescape15 abrt[6412]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:30 erogamescape15 abrt[6415]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:30 erogamescape15 abrtd: Directory 'ccpp-2013-04-13-12:16:28-7629' creation detected
Apr 13 12:16:30 erogamescape15 abrtd: Directory 'ccpp-2013-04-13-12:16:28-6139' creation detected
Apr 13 12:16:30 erogamescape15 abrt[6421]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:30 erogamescape15 abrt[6425]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:30 erogamescape15 abrt[6428]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:30 erogamescape15 abrt[6431]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:30 erogamescape15 abrt[6434]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:30 erogamescape15 abrt[6437]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:30 erogamescape15 abrt[6440]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:30 erogamescape15 abrt[6443]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:30 erogamescape15 abrt[6446]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:30 erogamescape15 abrtd: Executable '/usr/local/bin/pgpool' doesn't belong to any package
Apr 13 12:16:30 erogamescape15 abrtd: 'post-create' on '/var/spool/abrt/ccpp-2013-04-13-12:16:28-24018' exited with 1
Apr 13 12:16:30 erogamescape15 abrtd: Corrupted or bad directory '/var/spool/abrt/ccpp-2013-04-13-12:16:28-24018', deleting
Apr 13 12:16:30 erogamescape15 abrtd: Executable '/usr/local/bin/pgpool' doesn't belong to any package
Apr 13 12:16:30 erogamescape15 abrtd: 'post-create' on '/var/spool/abrt/ccpp-2013-04-13-12:16:28-7629' exited with 1
Apr 13 12:16:30 erogamescape15 abrtd: Corrupted or bad directory '/var/spool/abrt/ccpp-2013-04-13-12:16:28-7629', deleting
Apr 13 12:16:30 erogamescape15 abrtd: Executable '/usr/local/bin/pgpool' doesn't belong to any package
Apr 13 12:16:30 erogamescape15 abrtd: 'post-create' on '/var/spool/abrt/ccpp-2013-04-13-12:16:28-6139' exited with 1
Apr 13 12:16:30 erogamescape15 abrtd: Corrupted or bad directory '/var/spool/abrt/ccpp-2013-04-13-12:16:28-6139', deleting
Apr 13 12:16:31 erogamescape15 abrt[6453]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:31 erogamescape15 abrt[6460]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:31 erogamescape15 abrt[6461]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:31 erogamescape15 abrt[6466]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:31 erogamescape15 abrt[6469]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:31 erogamescape15 abrt[6472]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:31 erogamescape15 abrt[6475]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:31 erogamescape15 abrt[6478]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:31 erogamescape15 abrt[6480]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:31 erogamescape15 abrt[6484]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:31 erogamescape15 abrt[6485]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:31 erogamescape15 abrt[6490]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:31 erogamescape15 abrt[6493]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:31 erogamescape15 abrt[6496]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:31 erogamescape15 abrt[6499]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:31 erogamescape15 abrt[6502]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:31 erogamescape15 abrt[6503]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:31 erogamescape15 abrt[6508]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:32 erogamescape15 abrt[6511]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:32 erogamescape15 abrt[6522]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:33 erogamescape15 kernel: __ratelimit: 48 callbacks suppressed
Apr 13 12:16:33 erogamescape15 kernel: pgpool[11089]: segfault at 488 ip 000000000041866e sp 00007fffc4077cd0 error 4 in pgpool[400000+bf000]
Apr 13 12:16:33 erogamescape15 abrt[6525]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:33 erogamescape15 kernel: pgpool[7608]: segfault at 488 ip 000000000041866e sp 00007fffc4077d10 error 4 in pgpool[400000+bf000]
Apr 13 12:16:33 erogamescape15 abrt[6528]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:33 erogamescape15 kernel: pgpool[7512]: segfault at 488 ip 000000000041866e sp 00007fffc4077d10 error 4 in pgpool[400000+bf000]
Apr 13 12:16:33 erogamescape15 abrt[6531]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:33 erogamescape15 kernel: pgpool[5581]: segfault at 488 ip 000000000041866e sp 00007fffc4077cd0 error 4 in pgpool[400000+bf000]
Apr 13 12:16:33 erogamescape15 abrt[6534]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:33 erogamescape15 kernel: pgpool[12387]: segfault at 488 ip 000000000041866e sp 00007fffc4077cd0 error 4 in pgpool[400000+bf000]
Apr 13 12:16:33 erogamescape15 abrt[6536]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:33 erogamescape15 kernel: pgpool[12698]: segfault at 488 ip 000000000041866e sp 00007fffc4077cd0 error 4 in pgpool[400000+bf000]
Apr 13 12:16:33 erogamescape15 abrt[6538]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:33 erogamescape15 kernel: pgpool[7567]: segfault at 488 ip 000000000041866e sp 00007fffc4077d10 error 4 in pgpool[400000+bf000]
Apr 13 12:16:33 erogamescape15 abrt[6540]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:33 erogamescape15 kernel: pgpool[6159]: segfault at 488 ip 000000000041866e sp 00007fffc4077cd0 error 4 in pgpool[400000+bf000]
Apr 13 12:16:33 erogamescape15 abrt[6542]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:33 erogamescape15 kernel: pgpool[21900]: segfault at 488 ip 000000000041866e sp 00007fffc4077cd0 error 4 in pgpool[400000+bf000]
Apr 13 12:16:33 erogamescape15 abrt[6544]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:33 erogamescape15 kernel: pgpool[7644]: segfault at 488 ip 000000000041866e sp 00007fffc4077d10 error 4 in pgpool[400000+bf000]
Apr 13 12:16:33 erogamescape15 abrt[6546]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:33 erogamescape15 abrt[6549]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:34 erogamescape15 abrt[6551]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:34 erogamescape15 abrt[6553]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:34 erogamescape15 abrt[6557]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:34 erogamescape15 abrt[6559]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:34 erogamescape15 abrt[6561]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:34 erogamescape15 abrt[6563]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:34 erogamescape15 abrt[6565]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:34 erogamescape15 abrt[6568]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:34 erogamescape15 abrt[6570]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:34 erogamescape15 abrt[6571]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:34 erogamescape15 abrt[6574]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:35 erogamescape15 abrt[6578]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:35 erogamescape15 abrt[6582]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:35 erogamescape15 abrt[6586]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:35 erogamescape15 abrt[6588]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:35 erogamescape15 abrt[6590]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:35 erogamescape15 abrt[6593]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:35 erogamescape15 abrt[6595]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:35 erogamescape15 abrt[6597]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:35 erogamescape15 abrt[6598]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:35 erogamescape15 abrt[6601]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:35 erogamescape15 abrt[6603]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:35 erogamescape15 abrt[6605]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:35 erogamescape15 abrt[6607]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:35 erogamescape15 abrt[6609]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:35 erogamescape15 abrt[6610]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:35 erogamescape15 abrt[6613]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:36 erogamescape15 abrt[6619]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:38 erogamescape15 abrt[6636]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:38 erogamescape15 kernel: __ratelimit: 30 callbacks suppressed
Apr 13 12:16:38 erogamescape15 kernel: pgpool[7618]: segfault at 488 ip 000000000041866e sp 00007fffc4077d10 error 4 in pgpool[400000+bf000]
Apr 13 12:16:38 erogamescape15 abrt[6656]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:38 erogamescape15 kernel: pgpool[7682]: segfault at 488 ip 000000000041866e sp 00007fffc4077d10 error 4 in pgpool[400000+bf000]
Apr 13 12:16:38 erogamescape15 abrt[6658]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:38 erogamescape15 kernel: pgpool[3310]: segfault at 488 ip 000000000041866e sp 00007fffc4077cd0 error 4 in pgpool[400000+bf000]
Apr 13 12:16:38 erogamescape15 abrt[6660]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:39 erogamescape15 kernel: pgpool[6140]: segfault at 488 ip 000000000041866e sp 00007fffc4077cd0 error 4 in pgpool[400000+bf000]
Apr 13 12:16:39 erogamescape15 abrt[6662]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:39 erogamescape15 kernel: pgpool[7571]: segfault at 488 ip 000000000041866e sp 00007fffc4077d10 error 4 in pgpool[400000+bf000]
Apr 13 12:16:39 erogamescape15 abrt[6664]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:39 erogamescape15 kernel: pgpool[23677]: segfault at 488 ip 000000000041866e sp 00007fffc4077cd0 error 4 in pgpool[400000+bf000]
Apr 13 12:16:39 erogamescape15 abrt[6666]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:39 erogamescape15 kernel: pgpool[4231]: segfault at 488 ip 000000000041866e sp 00007fffc4077cd0 error 4 in pgpool[400000+bf000]
Apr 13 12:16:39 erogamescape15 abrt[6669]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:39 erogamescape15 kernel: pgpool[7661]: segfault at 488 ip 000000000041866e sp 00007fffc4077d10 error 4 in pgpool[400000+bf000]
Apr 13 12:16:39 erogamescape15 abrt[6671]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:39 erogamescape15 kernel: pgpool[7487]: segfault at 488 ip 000000000041866e sp 00007fffc4077cd0 error 4 in pgpool[400000+bf000]
Apr 13 12:16:39 erogamescape15 abrt[6673]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:39 erogamescape15 kernel: pgpool[1165]: segfault at 488 ip 000000000041866e sp 00007fffc4077cd0 error 4 in pgpool[400000+bf000]
Apr 13 12:16:39 erogamescape15 abrt[6675]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:39 erogamescape15 abrt[6677]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:39 erogamescape15 abrt[6679]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:39 erogamescape15 abrt[6681]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:39 erogamescape15 abrt[6683]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:40 erogamescape15 abrt[6685]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:40 erogamescape15 abrt[6687]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:40 erogamescape15 abrt[6689]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:40 erogamescape15 abrt[6691]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:40 erogamescape15 abrt[6693]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:40 erogamescape15 abrt[6697]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:40 erogamescape15 abrt[6699]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:40 erogamescape15 abrt[6701]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:41 erogamescape15 abrt[6704]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:41 erogamescape15 abrt[6707]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:41 erogamescape15 abrt[6709]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:41 erogamescape15 abrt[6711]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:41 erogamescape15 abrt[6714]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:41 erogamescape15 abrt[6716]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:41 erogamescape15 abrt[6719]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:41 erogamescape15 abrt[6720]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:42 erogamescape15 abrt[6726]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:42 erogamescape15 abrt[6729]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:43 erogamescape15 abrt[6733]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:43 erogamescape15 abrt[6738]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:43 erogamescape15 abrt[6740]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:43 erogamescape15 abrt[6743]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:43 erogamescape15 abrt[6745]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:43 erogamescape15 abrt[6747]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:43 erogamescape15 abrt[6749]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:43 erogamescape15 abrt[6752]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:43 erogamescape15 abrt[6754]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:43 erogamescape15 abrt[6755]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:44 erogamescape15 kernel: __ratelimit: 32 callbacks suppressed
Apr 13 12:16:44 erogamescape15 kernel: pgpool[7658]: segfault at 488 ip 000000000041866e sp 00007fffc4077d10 error 4 in pgpool[400000+bf000]
Apr 13 12:16:44 erogamescape15 abrt[6760]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:44 erogamescape15 kernel: pgpool[7625]: segfault at 488 ip 000000000041866e sp 00007fffc4077d10 error 4 in pgpool[400000+bf000]
Apr 13 12:16:44 erogamescape15 abrt[6762]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:45 erogamescape15 kernel: pgpool[26051]: segfault at 488 ip 000000000041866e sp 00007fffc4077cd0 error 4 in pgpool[400000+bf000]
Apr 13 12:16:45 erogamescape15 abrt[6764]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:46 erogamescape15 kernel: pgpool[7677]: segfault at 488 ip 000000000041866e sp 00007fffc4077d10 error 4 in pgpool[400000+bf000]
Apr 13 12:16:46 erogamescape15 abrt[6772]: Not saving repeating crash in '/usr/local/bin/pgpool'
Apr 13 12:16:46 erogamescape15 kernel: pgpool[7585]: segfault at 488 ip 000000000041866e sp 00007fffc4077d10 error 4 in pgpool[400000+bf000]
Apr 13 12:16:46 erogamescape15 abrt[6774]: Not saving repeating crash in '/usr/local/bin/pgpool'