Add pgbouncer authentication failure log format to pgsql-logs (#1881) by rafaelsoje · Pull Request #1951 · crowdsecurity/hub

rafaelsoje · 2026-10-18T21:19:39Z

Description

This PR adds parsing support for PgBouncer authentication failure log formats (both password authentication and SASL failures), closing #1881.

Details

Updates filter to accept both postgres and pgbouncer programs: evt.Parsed.program in ['postgres', 'pgbouncer'].
Adds grok pattern for PgBouncer ERROR C-... password authentication failed.
Adds grok pattern for PgBouncer LOG C-... closing because: SASL authentication failed.
Extracts timestamp, source_ip, source_port, pgsql_target_user, pgsql_dbname, and auth_method.
Emits log_type: pgsql_failed_auth, allowing automatic correlation with the existing crowdsecurity/pgsql-bf brute-force detection scenario.
Updated test logs and assertions in .tests/pgsql-logs/.

Closes #1881

Checklist

I have read the contributing guide
I have tested my changes locally
For new parsers or scenarios, tests have been added
I have run the hub linter and no issues were reported (see contributing guide)
Automated tests are passing
AI was used to generate any/all content of this PR

…ecurity#1881)

blotus · 2026-10-20T08:38:03Z

@@ -1,4 +1,4 @@
-len(results["s01-parse"]["crowdsecurity/pgsql-logs"]) == 6
+len(results["s01-parse"]["crowdsecurity/pgsql-logs"]) == 8


Looks like the assert file was only partially regenerated?
It's missing the full asserts for the 2 new lines

Done! Added the missing assertions for both PgBouncer log lines and verified that all tests are passing locally.

sabban

hi,

THank you for your contribution. Here's a review and please add a piece of doc to explain how to configure the datasource

sabban · 2026-10-28T15:03:51Z

+  - grok:
+    #pgbouncer SASL authentication failed
+      pattern: '%{TIMESTAMP_ISO8601:timestamp} %{WORD:zone} \[%{INT:PID}\] LOG C-%{NOTSPACE:connection_id}: %{DATA:pgsql_dbname}/%{DATA:pgsql_target_user}@%{IP:source_ip}:%{INT:source_port} closing because: %{WORD:auth_method} authentication failed'
+      apply_on: message


It seems that you are counting each failed login twice.

Fixed in ca7572d. I removed the duplicate grok pattern for connection closing. Only the initial ERROR log line is now parsed to prevent duplicate counting, and I added a test assertion verifying that closing lines are ignored.

sabban · 2026-10-28T15:18:41Z

      apply_on: message
+  - grok:
+    #pgbouncer password authentication failed
+      pattern: '%{TIMESTAMP_ISO8601:timestamp} %{WORD:zone} \[%{INT:PID}\] ERROR C-%{NOTSPACE:connection_id}: %{DATA:pgsql_dbname}/%{DATA:pgsql_target_user}@%{IP:source_ip}:%{INT:source_port} %{WORD:auth_method} authentication failed'


I believe this is not ipv6 compatible. You can use [? and ]? to add optional brackets

Fixed in ca7572d. I updated the pattern to @[?%{IP:source_ip}]?:%{INT:source_port} and added test cases verifying bracketed IPv6 parsing (@[2001:db8::1]:port).

sabban · 2026-10-28T15:43:48Z

-filter: "evt.Parsed.program == 'postgres'"
+description: "Parse PgSQL and PgBouncer logs"
+filter: "evt.Parsed.program in ['postgres', 'pgbouncer']"
 nodes:


why adding pgbouncer here as there's the logs you added still have postgres ?

I kept pgbouncer in the filter so both file acquisition (labels.type: pgbouncer) and syslog/journald (where program name is pgbouncer) are supported. I added a dedicated test suite under .tests/pgbouncer-logs/ with log_type: pgbouncer (testing IPv4, IPv6, ignoring close logs, and bruteforce scenario overflow), while restoring .tests/pgsql-logs/ to focus purely on PostgreSQL.

- Prevent double counting by removing duplicate closing because grok pattern - Support bracketed IPv6 addresses in PgBouncer grok pattern - Add dedicated pgbouncer-logs hub test with log_type pgbouncer - Add datasource configuration documentation in pgsql-logs.md

rafaelsoje · 2026-10-29T00:16:27Z

Hi @sabban,

Thank you for the detailed review! All requested points have been addressed in commit ca7572d:

Double Counting: I removed the grok pattern matching the connection close message (LOG C-... closing because: SASL authentication failed). Only the initial ERROR C-... <method> authentication failed log entry is now parsed, preventing duplicate event increments. I also added a test assertion verifying that closing lines are ignored.
IPv6 Compatibility: I updated the pattern to @\[?%{IP:source_ip}\]?:%{INT:source_port}. Verified with bracketed IPv6 addresses (@[2001:db8::1]:port), extracting the clean IP without brackets.
Filter and Test Consistency: I kept evt.Parsed.program in ['postgres', 'pgbouncer'] so both file acquisition (type: pgbouncer) and syslog/journald (where program name is pgbouncer) are supported. I added a dedicated test suite under .tests/pgbouncer-logs/ with log_type: pgbouncer (testing IPv4, IPv6, ignoring close logs, and bruteforce scenario overflow), while restoring .tests/pgsql-logs/ to focus purely on PostgreSQL.
Documentation: I updated parsers/s01-parse/crowdsecurity/pgsql-logs.md with explicit acquis.yaml examples for both PostgreSQL and PgBouncer (file and syslog/journald), along with a summary table of parsed fields.

All hub tests (pgsql-logs with 100 assertions and pgbouncer-logs with 106 assertions) pass with 100% success.

sabban

one last question before merging

sabban · 2026-11-03T09:40:57Z

+
+```yaml
+journalctl_filter:
+  - _SYSTEMD_UNIT=pgbouncer.service


did you you actually test this setup ?

Good catch, you are completely right. I had only validated the file-based configuration (type: pgbouncer on /var/log/postgresql/pgbouncer.log, which is tested under .tests/pgbouncer-logs), and added the journalctl snippet speculatively (which was also missing source: journalctl).

Since syslog/journald setups often format or strip application prefixes differently depending on daemon setup and haven't been validated end-to-end here, I removed the journalctl block so the documentation focuses strictly on the tested file-based acquisition, consistent with the PostgreSQL section.

Add pgbouncer authentication failure log format to pgsql-logs (crowds…

e0d7e9f

…ecurity#1881)

blotus requested changes Oct 20, 2026

View reviewed changes

Add missing assertions for pgbouncer logs in pgsql-logs

d02acb4

sabban requested changes Oct 28, 2026

View reviewed changes

rafaelsoje force-pushed the add-pgbouncer-auth-logs branch from f11eea9 to ca7572d Compare October 29, 2026 00:03

sabban reviewed Nov 3, 2026

View reviewed changes

remove untested journalctl acquisition snippet from pgsql-logs

baca760

sabban merged commit 58b38ff into crowdsecurity:master Nov 3, 2026
5 checks passed

		@@ -1,4 +1,4 @@
		len(results["s01-parse"]["crowdsecurity/pgsql-logs"]) == 6
		len(results["s01-parse"]["crowdsecurity/pgsql-logs"]) == 8

Conversation

rafaelsoje commented Oct 18, 2026

Description

Details

Checklist

Uh oh!

blotus Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

rafaelsoje Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

sabban left a comment

Choose a reason for hiding this comment

Uh oh!

sabban Oct 28, 2026

Choose a reason for hiding this comment

Uh oh!

rafaelsoje Oct 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sabban Oct 28, 2026

Choose a reason for hiding this comment

Uh oh!

rafaelsoje Oct 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sabban Oct 28, 2026

Choose a reason for hiding this comment

Uh oh!

rafaelsoje Oct 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rafaelsoje commented Oct 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sabban left a comment

Choose a reason for hiding this comment

Uh oh!

sabban Nov 3, 2026

Choose a reason for hiding this comment

Uh oh!

rafaelsoje Nov 3, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rafaelsoje Oct 29, 2026 •

edited

Loading

rafaelsoje Oct 29, 2026 •

edited

Loading

rafaelsoje Oct 29, 2026 •

edited

Loading

rafaelsoje commented Oct 29, 2026 •

edited

Loading