Postfix+Amavisd+Spamassassin+Clamav low performance on 14-host deployment

[Gianluca Marcari]

Hello,


We deployed a 14-host parallel postfix+amavis installation and are
having performance definitely not in line with what we were expecting
(and actually getting from a previos lower-scale installation).

Scenario:


Load balancer publishes a virtual IP which spreads the load on the 14
content filtering servers (plus the 5 non-filtering servers which were
previously handling the load, we had to bring them back into the group
because the 14 servers can't cope with the traffic)

The boxes have a postfix instance which performs some
RBL+header+black/whitelist checks and then forwards to the local
amavisd-new instance listening on the usual port 10024

Amavisd does an LDAP query against our centralized ldap, then performs
SpamAssassin + ClamAV checks

Viruses get destroyed
Spam gets destroyed unless the recipient is an AmavisVirusLover
(400.000 users out of 6 millions in the LDAP)
There is no quarantine at all, for anything (virus/spam/badheader/etc)

Everything not destroyed (ham, spam to AmavisVirusLovers) is sent to
the original 5 SMTP servers which handle it with no apparent proble
(If we shunt all the traffic to the 5 original SMTP servers,
everything flows smoothly).

Boxes are 2x2.66Ghz HT Xeons, 1GB Ram, 2 Ultra-320 HDDs in hardware
RAID1, Linux 2.6.12.3, amavis has its $TEMPBASE on a ramdrive
(/dev/shm).

The systems are processing between 7 and 14 emails per second, while
receiving up to 40, thus falling behind constantly.


Other relevant data:
Perl v5.8.5
Logging via syslog
Filesystems are reiserfs v3
SpamAssassin version 3.0.4



Relevant data in /etc/amavisd:conf:
$max_servers = 10;
$max_requests = 10;
$enable_ldap = 1;


Even though there is only 1 (mirrored) disk, we are seeing almost no
iowait (2% max, often 0%). System load is split between 74%usr, 22%
sys and 3% idle (fairly constant on most boxes)

From our logs, many mails get processed and handed over to the backend
in 300-500 milliseconds, but some are in the 5000-150000 milliseconds
range:

Sep 1 08:35:12 asav1 amavis[1224]: (01224-02-3) TIMING [total 7626 ms]
Sep 1 08:35:12 asav1 amavis[1299]: (01299-02-4) TIMING [total 5853 ms]
Sep 1 08:35:13 asav1 amavis[1720]: (01720-02-4) TIMING [total 5570 ms]
Sep 1 08:35:13 asav1 amavis[1592]: (01592-01-10) TIMING [total 417 ms]
Sep 1 08:35:13 asav1 amavis[1926]: (01926-01-4) TIMING [total 5485 ms]
Sep 1 08:35:13 asav1 amavis[1566]: (01566-01-10) TIMING [total 5504 ms]
Sep 1 08:35:13 asav1 amavis[1777]: (01777-01-4) TIMING [total 5550 ms]
Sep 1 08:35:13 asav1 amavis[1224]: (01224-02-4) TIMING [total 1050 ms]
Sep 1 08:35:13 asav1 amavis[1200]: (01200-02-10) TIMING [total 5186 ms]
Sep 1 08:35:14 asav1 amavis[1566]: (01566-01-11) TIMING [total 610 ms]
Sep 1 08:35:14 asav1 amavis[1224]: (01224-02-5) TIMING [total 811 ms]
Sep 1 08:35:14 asav1 amavis[1200]: (01200-02-11) TIMING [total 677 ms]
Sep 1 08:35:14 asav1 amavis[1720]: (01720-02-5) TIMING [total 1829 ms]
Sep 1 08:35:14 asav1 amavis[1299]: (01299-02-5) TIMING [total 2206 ms]
Sep 1 08:35:15 asav1 amavis[1921]: (01921-01-10) TIMING [total 5537 ms]
Sep 1 08:35:15 asav1 amavis[1394]: (01394-02-4) TIMING [total 5678 ms]
Sep 1 08:35:16 asav1 amavis[1720]: (01720-02-6) TIMING [total 1436 ms]
Sep 1 08:35:16 asav1 amavis[1921]: (01921-01-11) TIMING [total 1265 ms]
Sep 1 08:35:16 asav1 amavis[1969]: (01969-01-7) TIMING [total 6756 ms]
Sep 1 08:35:16 asav1 amavis[1566]: (01566-02) TIMING [total 2813 ms]
Sep 1 08:35:17 asav1 amavis[2093]: (02093-01) TIMING [total 5355 ms]
Sep 1 08:35:17 asav1 amavis[1885]: (01885-01-11) TIMING [total 5401 ms]
Sep 1 08:35:17 asav1 amavis[1238]: (01238-02-5) TIMING [total 5412 ms]
Sep 1 08:35:17 asav1 amavis[2093]: (02093-01-2) TIMING [total 481 ms]
Sep 1 08:35:18 asav1 amavis[1969]: (01969-01-8) TIMING [total 1298 ms]
Sep 1 08:35:18 asav1 amavis[2149]: (02149-01) TIMING [total 2717 ms]
Sep 1 08:35:18 asav1 amavis[1238]: (01238-02-6) TIMING [total 1021 ms]
Sep 1 08:35:18 asav1 amavis[1885]: (01885-02) TIMING [total 1090 ms]
Sep 1 08:35:19 asav1 amavis[1885]: (01885-02-2) TIMING [total 813 ms]
Sep 1 08:35:19 asav1 amavis[1720]: (01720-02-7) TIMING [total 3098 ms]
Sep 1 08:35:19 asav1 amavis[1969]: (01969-01-9) TIMING [total 1437 ms]
Sep 1 08:35:19 asav1 amavis[1238]: (01238-02-7) TIMING [total 1184 ms]
Sep 1 08:35:19 asav1 amavis[1777]: (01777-01-5) TIMING [total 6222 ms]
Sep 1 08:35:19 asav1 amavis[1299]: (01299-02-6) TIMING [total 4772 ms]
Sep 1 08:35:19 asav1 amavis[1592]: (01592-01-11) TIMING [total 6732 ms]
Sep 1 08:35:19 asav1 amavis[1969]: (01969-01-10) TIMING [total 279 ms]



Mail which have long processing time exhibit an odd breakdown,
spending a huge chunk of that time in fwd-rcpt-to and sometimes
best_try_originator


Sep 1 08:37:16 asav1 amavis[3682]: (03682-02-4) TIMING [total 5382
ms] - lookup_ldap: 16 (0%)0, SMTP pre-DATA-flush: 1 (0%)0, SMTP DATA:
66 (1%)2, body_digest: 1 (0%)2, gen_mail_id: 0 (0%)2, mime_decode: 33
(1%)2, get-file-type4: 16 (0%)2, decompose_part: 1 (0%)3,
decompose_part: 3 (0%)3, get-file-type1: 14 (0%)3, decompose_part: 57
(1%)4, parts_decode: 0 (0%)4, AV-scan-1: 10 (0%)4,
read_snmp_variables: 1 (0%)4, best_try_originator: 7 (0%)4,
update_cache: 1 (0%)4, fwd-connect: 24 (0%)5, fwd-mail-from: 2 (0%)5,
fwd-rcpt-to: 5094 (95%)99, fwd-rundown: 3 (0%)99, deal_with_mail_size:
1 (0%)99, main_log_entry: 27 (1%)100, update_snmp: 2 (0%)100,
unlink-4-files: 1 (0%)100, rundown: 0 (0%)100

Sep 1 08:37:16 asav1 amavis[4051]: (04051-01-5) TIMING [total 5437
ms] - lookup_ldap: 12 (0%)0, SMTP pre-DATA-flush: 1 (0%)0, SMTP DATA:
69 (1%)2, body_digest: 1 (0%)2, gen_mail_id: 0 (0%)2, mime_decode: 17
(0%)2, get-file-type2: 14 (0%)2, decompose_part: 1 (0%)2,
decompose_part: 51 (1%)3, parts_decode: 0 (0%)3, best_try_originator:
9 (0%)3, update_cache: 35 (1%)4, fwd-connect: 22 (0%)4, fwd-mail-from:
1 (0%)4, fwd-rcpt-to: 5158 (95%)99, fwd-rundown: 4 (0%)99,
deal_with_mail_size: 1 (0%)99, main_log_entry: 34 (1%)100,
update_snmp: 3 (0%)100, unlink-2-files: 2 (0%)100, rundown: 1 (0%)100

Sep 1 08:37:16 asav1 amavis[3425]: (03425-01-9) TIMING [total 5407
ms] - lookup_ldap: 33 (1%)1, SMTP pre-DATA-flush: 2 (0%)1, SMTP DATA:
12 (0%)1, body_digest: 1 (0%)1, gen_mail_id: 0 (0%)1, mime_decode: 70
(1%)2, get-file-type2: 30 (1%)3, decompose_part: 1 (0%)3,
decompose_part: 43 (1%)4, parts_decode: 0 (0%)4, AV-scan-1: 9 (0%)4,
read_snmp_variables: 1 (0%)4, best_try_originator: 21 (0%)4,
update_cache: 1 (0%)4, fwd-connect: 15 (0%)4, fwd-mail-from: 2 (0%)4,
fwd-rcpt-to: 5135 (95%)99, fwd-rundown: 4 (0%)100,
deal_with_mail_size: 1 (0%)100, main_log_entry: 21 (0%)100,
update_snmp: 2 (0%)100, unlink-2-files: 2 (0%)100, rundown: 1 (0%)100

Sep 1 08:37:19 asav1 amavis[3580]: (03580-02-3) TIMING [total 5032
ms] - lookup_ldap: 7 (0%)0, lookup_ldap: 5 (0%)0, lookup_ldap: 5
(0%)0, lookup_ldap: 6 (0%)0, lookup_ldap: 6 (0%)1, lookup_ldap: 6
(0%)1, lookup_ldap: 13 (0%)1, lookup_ldap: 17 (0%)1, lookup_ldap: 7
(0%)1, lookup_ldap: 36 (1%)2, lookup_ldap: 109 (2%)4, lookup_ldap: 100
(2%)6, lookup_ldap: 57 (1%)7, lookup_ldap: 46 (1%)8, lookup_ldap: 38
(1%)9, lookup_ldap: 60 (1%)10, lookup_ldap: 23 (0%)11, lookup_ldap: 45
(1%)12, lookup_ldap: 23 (0%)12, lookup_ldap: 9 (0%)12, lookup_ldap: 11
(0%)13, lookup_ldap: 14 (0%)13, lookup_ldap: 7 (0%)13, lookup_ldap: 8
(0%)13, lookup_ldap: 8 (0%)13, lookup_ldap: 7 (0%)13, lookup_ldap: 8
(0%)14, lookup_ldap: 17 (0%)14, lookup_ldap: 21 (0%)14, lookup_ldap:
16 (0%)15, SMTP pre-DATA-flush: 6 (0%)15, SMTP DATA: 42 (1%)16,
body_digest: 1 (0%)16, gen_mail_id: 1 (0%)16, mime_decode: 28 (1%)16,
get-file-type2: 14 (0%)16, parts_decode: 0 (0%)16, AV-scan-1: 52
(1%)18, spam-wb-list: 121 (2%)20, SA msg read: 5 (0%)20, ...



Sorry for the big mail, I tried to strike a balance between providing
enough useful information to troubleshoot the issue and flooding the
list.

Many thanks for any help you may provide!

Ciao,
Gianluca Marcari


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[Stephen Carter]

Make sure that the smtpd's which get the mail back from amavisd are
localhost:10026
inet n - - - - smtpd
-o smtpd_authorized_xforward_hosts=127.0.0.0/8
-o smtpd_client_restrictions=
-o smtpd_helo_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o mynetworks=127.0.0.0/8
-o >receive_override_options=no_unknown_recipient_checks,no_header_body_checks
-o content_filter=
thus preventing unneeded lookups!
--
Charite - Universitätsmedizin Berlin Tel. [phone]
Gemeinsame Einrichtung von FU- und HU-Berlin Fax. [phone]

I asked previously on the list if re-injection back into postfix would cause
postfix to re-check the e-mail based on smtpd settings configured in
main.cf file and I was told it would not.

Following this I turned on debugging in postfix, ran an e-mail through and
it appeared it was only checked once, on it's way in on port 25, and not
when it was re-injected from amavisd-new.

Although myself and the person who advised this may be wrong, it may be
worth testing by turning on debugging in postfix to confirm, as it may
end up being an unnecessary change.

Just a thought... no disrepect to Ralf's comments.

SteveC



-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[Gary V]

Post by Stephen Carter

Make sure that the smtpd's which get the mail back from amavisd are
localhost:10026
inet n - - - - smtpd
-o smtpd_authorized_xforward_hosts=127.0.0.0/8
-o smtpd_client_restrictions=
-o smtpd_helo_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o mynetworks=127.0.0.0/8
-o >receive_override_options=no_unknown_recipient_checks,no_header_body_checks
-o content_filter=
thus preventing unneeded lookups!
--
Charite - Universitätsmedizin Berlin Tel. [phone]
Gemeinsame Einrichtung von FU- und HU-Berlin Fax. [phone]

I asked previously on the list if re-injection back into postfix would cause
postfix to re-check the e-mail based on smtpd settings configured in
main.cf file and I was told it would not.
Following this I turned on debugging in postfix, ran an e-mail through and
it appeared it was only checked once, on it's way in on port 25, and not
when it was re-injected from amavisd-new.
Although myself and the person who advised this may be wrong, it may be
worth testing by turning on debugging in postfix to confirm, as it may
end up being an unnecessary change.
Just a thought... no disrepect to Ralf's comments.
SteveC

I think it is more common to use the setup listed in
http://www.ijs.si/software/amavisd/README.postfix.txt

127.0.0.1:10025 inet n - n - - smtpd
-o content_filter=
-o local_recipient_maps=
-o relay_recipient_maps=
-o smtpd_restriction_classes=
-o smtpd_delay_reject=no
-o smtpd_client_restrictions=permit_mynetworks,reject
-o smtpd_helo_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o mynetworks_style=host
-o mynetworks=127.0.0.0/8
-o strict_rfc821_envelopes=yes
-o smtpd_client_connection_count_limit=0
-o smtpd_client_connection_rate_limit=0
-o receive_override_options=no_header_body_checks,no_unknown_recipient_checks

I am curious how SpamAssassin is called, I don't see any SA related
timings in the TIMING examples. I think if you added more RAM, you
could increase the number of processes, and concurrently, the size of
the ramdrive.

Gary V



-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[Ralf Hildebrandt]

Post by Gianluca Marcari
$max_servers = 10;

Maybe this can be increased.

Post by Gianluca Marcari
$max_requests = 10;

This is a bit low.

Post by Gianluca Marcari
Mail which have long processing time exhibit an odd breakdown,
spending a huge chunk of that time in fwd-rcpt-to and sometimes
best_try_originator

Make sure that the smtpd's which get the mail back from amavisd are
configured like this:

localhost:10026
inet n - - - - smtpd
-o smtpd_authorized_xforward_hosts=127.0.0.0/8
-o smtpd_client_restrictions=
-o smtpd_helo_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o mynetworks=127.0.0.0/8
-o receive_override_options=no_unknown_recipient_checks,no_header_body_checks
-o content_filter=

thus preventing unneeded lookups!
--
Ralf Hildebrandt (i.A. des IT-Zentrums) ***@charite.de
Charite - Universitätsmedizin Berlin Tel. [phone]
Gemeinsame Einrichtung von FU- und HU-Berlin Fax. [phone]
IT-Zentrum Standort CBF send no mail to ***@charite.de


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[Gary V]

Post by Gianluca Marcari
The boxes have a postfix instance which performs some
RBL+header+black/whitelist checks and then forwards to the local
amavisd-new instance listening on the usual port 10024

Header/body checks are expensive. If nothing else, make sure they are
disabled on the reinjection side. Weigh their value carefully.

-o receive_override_options=no_header_body_checks,no_unknown_recipient_checks

Gary V



-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[Clifton Royston]

Post by Gianluca Marcari
We deployed a 14-host parallel postfix+amavis installation and are
having performance definitely not in line with what we were expecting
(and actually getting from a previos lower-scale installation).
Load balancer publishes a virtual IP which spreads the load on the 14
content filtering servers (plus the 5 non-filtering servers which were
previously handling the load, we had to bring them back into the group
because the 14 servers can't cope with the traffic)
The boxes have a postfix instance which performs some
RBL+header+black/whitelist checks and then forwards to the local
amavisd-new instance listening on the usual port 10024
Amavisd does an LDAP query against our centralized ldap, then performs
SpamAssassin + ClamAV checks
Viruses get destroyed
Spam gets destroyed unless the recipient is an AmavisVirusLover
(400.000 users out of 6 millions in the LDAP)
There is no quarantine at all, for anything (virus/spam/badheader/etc)
Everything not destroyed (ham, spam to AmavisVirusLovers) is sent to
the original 5 SMTP servers which handle it with no apparent proble
(If we shunt all the traffic to the 5 original SMTP servers,
everything flows smoothly).
Boxes are 2x2.66Ghz HT Xeons, 1GB Ram, 2 Ultra-320 HDDs in hardware
RAID1, Linux 2.6.12.3, amavis has its $TEMPBASE on a ramdrive
(/dev/shm).
The systems are processing between 7 and 14 emails per second, while
receiving up to 40, thus falling behind constantly.
Perl v5.8.5
Logging via syslog
Filesystems are reiserfs v3
SpamAssassin version 3.0.4
$max_servers = 10;
$max_requests = 10;
$enable_ldap = 1;

My calculations based on the above are that on a comparable system,
excluding LDAP, you should be handling about 4.1 msgs/sec steady load
through each of the scanners, hence 57 msgs/sec throughput for the
cluster of 14. (This is based on my numbers for dual 3GHz Xeons on
FreeBSD.)

Post by Gianluca Marcari
Even though there is only 1 (mirrored) disk, we are seeing almost no
iowait (2% max, often 0%). System load is split between 74%usr, 22%
sys and 3% idle (fairly constant on most boxes)

Under load you should see virtually 0 idle time. If all free CPU is
not being soaked up by SA PCRE matching, something is suboptimal.

Are you running a local DNS cache on each machine? I'd install the
DJB tinydns dnscache process on every one (configured as a forwarder to
your main DNS server.) You might also raise the number of amavisd
processes slightly (and boost max_requests a lot, e.g. to 50 or 100, so
it's not constantly killing/respawning children under heavy load.)

Post by Gianluca Marcari

Post by Gianluca Marcari
From our logs, many mails get processed and handed over to the backend

in 300-500 milliseconds, but some are in the 5000-150000 milliseconds

The former is right, the latter indicates something screwy is going
on.

...

Post by Gianluca Marcari
Mail which have long processing time exhibit an odd breakdown,
spending a huge chunk of that time in fwd-rcpt-to and sometimes
best_try_originator
Sep 1 08:37:16 asav1 amavis[3682]: (03682-02-4) TIMING [total 5382
66 (1%)2, body_digest: 1 (0%)2, gen_mail_id: 0 (0%)2, mime_decode: 33
(1%)2, get-file-type4: 16 (0%)2, decompose_part: 1 (0%)3,
decompose_part: 3 (0%)3, get-file-type1: 14 (0%)3, decompose_part: 57
(1%)4, parts_decode: 0 (0%)4, AV-scan-1: 10 (0%)4,
read_snmp_variables: 1 (0%)4, best_try_originator: 7 (0%)4,
update_cache: 1 (0%)4, fwd-connect: 24 (0%)5, fwd-mail-from: 2 (0%)5,
1 (0%)99, main_log_entry: 27 (1%)100, update_snmp: 2 (0%)100,
unlink-4-files: 1 (0%)100, rundown: 0 (0%)100

There are a lot of possibilities here, but it's all happening in the
handoff back to Postfix. I think this could be indicating LDAP lookup
delays *or* inappropriate DNS check delays on the receiving Postfix
process. Look at load, concurrent requests, etc. on your LDAP server.
Look at the Postfix logs for reinjection processes, see if you can get
timing information on that side for the slow transactions and if it
tallies with what you see in the amavisd timing breakdown.

...

Post by Gianluca Marcari
Sorry for the big mail, I tried to strike a balance between providing
enough useful information to troubleshoot the issue and flooding the
list.

No problem, this was perfect. Thanks for providing so much relevant
information, and I hope somebody can point you to where the issue is!

-- Clifton
--
Clifton Royston -- ***@tikitechnologies.com
Tiki Technologies Lead Programmer/Software Architect
"My own personal theory is that this is the very dawn of the world.
We're hardly more than an eyeblink away from the fall of Troy, and
scarcely an interglaciation removed from the Altamira cave painters. We
live in extremely interesting ancient times.
I like this idea. It encourages us to be earnest and ingenious and
brave, as befits ancestral peoples; but keeps us from deciding that
because we don't know all the answers, they must be unknowable and thus
unprofitable to pursue." -- Teresa Nielsen Hayden, 1995


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[Mark Martinec]

Gianluca,

Post by Gianluca Marcari
The boxes have a postfix instance which performs some
RBL+header+black/whitelist checks and then forwards to the local
amavisd-new instance listening on the usual port 10024

Make sure these RBL and DNS checks are not done again by
a smtpd Postfix service on port 10025.

Post by Gianluca Marcari
System load is split between 74%usr, 22%
sys and 3% idle (fairly constant on most boxes)
Mail which have long processing time exhibit an odd breakdown,
spending a huge chunk of that time in fwd-rcpt-to and sometimes
best_try_originator
fwd-connect: 24 (0%)5, fwd-mail-from: 2 (0%)5, fwd-rcpt-to: 5094 (95%)99,
fwd-rundown: 3 (0%)99,

There is certainly something fishy there on the Postfix side,
taking 5 seconds to respond to a RCPT TO command.
On the other hand, the 3% idle indicates that even if Postfix
responded faster, the mail throughput probably wouldn't
be increased by much.

What is interesting is that in all of your shown log entries
the initial fwd-* sections weren't followed by an actual mail
transfer (write-header, fwd-data, fwd-data-end). This indicates
that most likely these messages were actually rejected by
Postfix smtpd on port 10025, probably because of some
client/sender/recipient restriction (not header or body check,
these come later).

Check that you have all checks that were already performed once
on port 25, turned off on porty 10025. Use local caching-only DNS
server on each host. Try to match these events from the amavis log
to those on the Postfix log.

Mark


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[Mark Martinec]

Use local caching-only DNS server on each host.

On a second though, perhaps not.
One or two nearby lightly loaded hosts providing a caching DNS server
could probably deal better with 14 client hosts, than running dns server
on each heavily loaded host with only 3% idle time.

Mark


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[Gary V]

Sep 1 08:37:16 asav1 amavis[4051]: (04051-01-5) TIMING [total 5437
ms] - lookup_ldap: 12 (0%)0, SMTP pre-DATA-flush: 1 (0%)0, SMTP DATA:
69 (1%)2, body_digest: 1 (0%)2, gen_mail_id: 0 (0%)2, mime_decode: 17
(0%)2, get-file-type2: 14 (0%)2, decompose_part: 1 (0%)2,
decompose_part: 51 (1%)3, parts_decode: 0 (0%)3, best_try_originator:
9 (0%)3, update_cache: 35 (1%)4, fwd-connect: 22 (0%)4, fwd-mail-from:
1 (0%)4, fwd-rcpt-to: 5158 (95%)99, fwd-rundown: 4 (0%)99,
deal_with_mail_size: 1 (0%)99, main_log_entry: 34 (1%)100,
update_snmp: 3 (0%)100, unlink-2-files: 2 (0%)100, rundown: 1 (0%)100

Since I don't see any SA timings, I'm really interested how spamassassin
is called. If I were to guess, I would say something of this nature:

spamchk unix - n n - 10 pipe
flags=Rq user=filter argv=/usr/local/bin/spamchk -f ${sender} -- ${recipient}

127.0.0.1:10025 inet n - n - - smtpd
-o content_filter=spamchk:dummy

This is from:
http://www.akadia.com/services/postfix_spamassassin.html

This is all pure conjecture, but could something like this explain
"fwd-rcpt-to: 5158 (95%)"?

Gary V



-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[KISHOR.MV]

Hello all,

I have sendmail dual-mta setup.
I am using sam from horde to setup and sql lookup for amavisd for per user
settings on top of the global settings.
Now my users prefer that if they setup the spam filtering in horde to move
the spam taged messages to a new folder. This effects when the user applies
the settings when they list the inbox the filter rules move the messages to
the custom spam folder.

Can this be integerated when during the mail delivery at the spam filtering
itself that no matter what email client they use, either horde or pop3
clients when the user set the spam filtering option in horde to move the
messages to a different folder that spam filter from amavis take the
settings from mysql table and deliver the messages to the particular folder
based on the user settings.

Or is there a way to deliver the messages to a common folder name under
individual user folders?

regards
Kishor



-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[l***@kwsoft.de]

Post by KISHOR.MV
Hello all,
I have sendmail dual-mta setup.

Never ever hijack someone others thread to ask if you want a useful answer at
all. If you are lazy copy and paste the list-address but hitting answer and
start a new question is not a option at all.

Andreas




-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[Gary V]

Post by Gary V
Since I don't see any SA timings, I'm really interested how spamassassin
is called.

I have no idea if Gianluca is using spamd/spamc or not but:

I was personally curious about the efficiency of using spamc
outside of amavisd-new, compared to having amavisd-new call SA.
I did an anecdotal test by feeding a test message a number of
times through spamc:

time spamc -u amavis < sample-spam.txt
and it was fairly consistent at around 14 seconds. (800Mhz Celeron)

Sending the same messages through amavisd-new a few times showed SA
check at 2.4 seconds, dropping to .7 seconds at times.

I then installed a local dns cache (djbdns) and tried the tests again.
spamc dropped to around 2-4 seconds (decreased with the second test),
and SA check within amavisd-new was around 2 seconds, dropping to .7
seconds at times.

BTW, much to my surprise, I also noticed that 'SA check:' does not
necessarily appear in every entry. I imagine this due to caching. So
I would assume that at times, the spam check is free. This of course
could explain why I did not see any SA checks in Gianluca's post.

I wouldn't absolutely draw conclusions from this, but it appears to me
it's a good idea to point your resolver at a caching DNS server, and
amavisd-new uses spamassassin more efficiently than spamd/spamc. Not
even counting the fact that within amavisd-new, SpamAssassin is called
only once for each message regardless of the number of recipients.

Gary V



-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[Bojan Zdrnja]

Hi Gianluca,

-----Original Message-----
Gianluca Marcari
Sent: Friday, 2 September 2005 10:35 p.m.
Subject: [AMaViS-user] Postfix+Amavisd+Spamassassin+Clamav
low performance on 14-host deployment
Hello,
We deployed a 14-host parallel postfix+amavis installation and are
having performance definitely not in line with what we were expecting
(and actually getting from a previos lower-scale installation).

Impressive deployment :) I have only 4 servers in parallel ;-)

Boxes are 2x2.66Ghz HT Xeons, 1GB Ram, 2 Ultra-320 HDDs in hardware
RAID1, Linux 2.6.12.3, amavis has its $TEMPBASE on a ramdrive
(/dev/shm).

I would add more memory in these servers. My servers, for example, have 2.5
GB RAM each.
With more memory you might be able to increase number of amavisd-new
processes running in parallel, which could give you more throughput.

The systems are processing between 7 and 14 emails per second, while
receiving up to 40, thus falling behind constantly.

Processing sounds relatively ok to me.

Perl v5.8.5
Logging via syslog

You did remember to prefix entries with the "-" sign to omit syncing, right?
That can cause huge slowdowns.

Sep 1 08:37:16 asav1 amavis[3682]: (03682-02-4) TIMING [total 5382

This is unusually high. As other people said, it looks like your 2nd postfix
instance is doing some checks which take quite a bit. Be sure to disable all
unnecessary checks here.

Also, as Clifton said, it is good to install a local caching DNS.

Finally, for SpamAssassin, be sure to use MySQL storage backend and not
BerkeleyDB.

Cheers,

Bojan

--
Bojan Zdrnja, CISSP, RHCE
Security Implementation Specialist
Information Technology Systems and Services (ITSS)
The University of Auckland, New Zealand



-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[Michael Scheidell]

-----Original Message-----
Mark Martinec
Sent: Friday, September 02, 2005 8:34 PM
Subject: Re: [AMaViS-user]
Postfix+Amavisd+Spamassassin+Clamav low performance on
14-host deployment

Use local caching-only DNS server on each host.

On a second though, perhaps not.
One or two nearby lightly loaded hosts providing a caching
DNS server could probably deal better with 14 client hosts,
than running dns server on each heavily loaded host with only
3% idle time.

Especially in a load balanced situation where it is likely that all the
hosts are looking up all the same domains for 'dictionary attack runs'
--
Michael Scheidell, CTO
561-999-5000, ext 1131
SECNAP Network Security Corporation
Keep up to date with latest information on IT security:
Real time security alerts: http://www.secnap.com/news



-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[Bojan Zdrnja]

-----Original Message-----
Michael Scheidell
Sent: Sunday, 4 September 2005 12:13 a.m.
Subject: RE: [AMaViS-user]
Postfix+Amavisd+Spamassassin+Clamav low performance on
14-host deployment

Post by Mark Martinec

Use local caching-only DNS server on each host.

On a second though, perhaps not.
One or two nearby lightly loaded hosts providing a caching
DNS server could probably deal better with 14 client hosts,
than running dns server on each heavily loaded host with only
3% idle time.

Especially in a load balanced situation where it is likely
that all the
hosts are looking up all the same domains for 'dictionary attack runs'

I still think that the majority of DNS requests will be answered by the
local (caching) DNS server. Besides, DNS requests going out on the NIC are
more expensive then those going on the loopback interface.
From my experience in the real world (YMMV, of course), locally installed
DNS server offers more benefits than remote one (remote on the local
network, of course).
If would be maybe worth doing a performance benchmark to see which setup
behaves better.

Anyway, that's not the cause why the OP has long delays (something else is
failing).

Cheers,

Bojan



-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[a***@mikecappella.com]

Post by Bojan Zdrnja
I still think that the majority of DNS requests will be
answered by the local (caching) DNS server. Besides, DNS
requests going out on the NIC are more expensive then those
going on the loopback interface.

This is probably true in a vacuum, but I think if you're going to compare
the two, you also need to factor process context switches and hindrance to
other parts of the system. Running a local caching server means that the
server is likely being used by the system for other things too, and this
means frequent interruptions to your mail services, or if your mail services
are CPU hogging, other DNS lookups on the same server wait until your mail
services complete their full process quantum. It's all a balance, and
sometimes running something locally can be a hindrance vs. running a
dedicated remote service. Consider that you can allocate LOTS of RAM to a
dedicated DNS server, where you probably would not desire having a local DNS
service compete for resources used for mail services. The NIC will also DMA
process and handle incoming/outgoing packets while the system is doing other
mail processing, so there's some overlap.

But you're right, YMMV and benchmarks are the only way to be sure.



-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[Gianluca Marcari]

Sorry, I didn't send this to the list; resending

First of all, thanks everybody for overwhelming me with helpful pointers!

A quick update:

We traced the unusually high response times in the fwd-rcpt-to phase
to a side effect of our deployment: we were swamping the LDAP
directory with hundreds of queries per second, and the back-end SMTP
(which btw is not Postfix but a commercial one, my bad for forgetting
to write that (but the hints did apply!)) was trying to access the
same LDAP directory to verify local usernames).

Basically, we DoS'ed our own directory, thus severely hampering our
own SMTP backend, which showed up in the Amavisd timing as
fwd-rcpt-to.

As a brutal workaround we exported the list of spamlovers into a text
file and stopped querying the LDAP directories. Another team is
setting up a couple of additional LDAP slaves for our queries. Also, I
noticed we run LDAP queries for *every* recipient, even those not
belonging to our 5 domains. I didn't find any clue in the
documentation on how to disable this, but will keep looking.

Also, following hints in the list and on available literature, we
moved the postfix instances away on 5 Solaris machines which were
previously hosting the front-end custom SMTP instances and this is
definitely giving us a more predictable behaviour: no more hitting
swapfiles or I/O trashing and a much better load levelling on the 14
boxes (previously the balancer only saw the postfix instances, which
were very quick at accepting and queueing mail, so all the servers
appeared to be equally responsive and the load was not adaptive. Now,
with an Amavis virtualhost spreading traffic to the 14 10024 ports,
the balancer has much better info about system load and every new
session is opened against the least loaded server.).

The new setup is giving us some postfix issues (apparently we need
some tuning there as well, we temporarily turned off logging via
Solaris' Syslog which was killing the performances) but we are taking
care of it.

On the Amavis side, we brought up $max_servers and $max_requests
(at 20 atm, we are experimenting other values to find the sweet spot
for our setup) and have upgraded 6 machines to 2GB to run a
comparison.

The timing breakdown in the logs is now much more reasonable, total
processing time averages 400ms and is never over 2000 ms (except for a
few huge mails, but that's expected) and most of the time is now
accounted to SA check and AV-scan as we expected.

We are running a few load and stress tests but we'll probably have to
bring 16 more servers in the pool, since our peak load can be 24000
mails per minute.

We will keep tweaking the Amavis configuration because even with 30
servers this will amount to 13 emails/sec sustained and the servers
might not be able to service all requests satisfactorily.

Thanks everybody on the list for your suggestions, and please if you
have more don't hesitate to let me know :)


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[Ralf Hildebrandt]

Post by Gianluca Marcari
As a brutal workaround we exported the list of spamlovers into a text
file and stopped querying the LDAP directories. Another team is
setting up a couple of additional LDAP slaves for our queries. Also, I
noticed we run LDAP queries for *every* recipient, even those not
belonging to our 5 domains. I didn't find any clue in the
documentation on how to disable this, but will keep looking.

This is documented. Postfix needs to query the LDAP servers to know if
the domain is local, virtual, relay or none of these. For every
recipient! Thus it makes sense to keep virtual_alias_domains,
relay_domains and mydestination in a file instead of a LDAP server.
--
Ralf Hildebrandt (i.A. des IT-Zentrums) ***@charite.de
Charite - Universitätsmedizin Berlin Tel. [phone]
Gemeinsame Einrichtung von FU- und HU-Berlin Fax. [phone]
IT-Zentrum Standort CBF send no mail to ***@charite.de


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[Gianluca Marcari]

Post by Ralf Hildebrandt
This is documented. Postfix needs to query the LDAP servers to know if
the domain is local, virtual, relay or none of these. For every
recipient! Thus it makes sense to keep virtual_alias_domains,
relay_domains and mydestination in a file instead of a LDAP server.

Sorry, I was not clear, I meant amavis, not postfix. Postfix is not
configured for LDAP queries at the moment. Amavis was (and was
swamping the directory) in order to get the SpamLover attribute, and
now isn't (since we exported the spamlovers list into a flat file).

The odd behaviour is that Amavis, even though it knew our list of 5
local domains, kept querying the directory for every recipient, even
those not local to our systems.

I didn't find a policy bank based on recipient, so I'm looking for
other options in order to change this behaviour.

Post by Ralf Hildebrandt
Charite - Universitätsmedizin Berlin Tel. [phone]
Gemeinsame Einrichtung von FU- und HU-Berlin Fax. [phone]

Ciao,
Gianluca Marcari


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[Mark Martinec]

Gianluca,

Post by Gianluca Marcari

Post by Gianluca Marcari
We traced the unusually high response times in the fwd-rcpt-to phase
to a side effect of our deployment: we were swamping the LDAP
directory with hundreds of queries per second, and the back-end SMTP
(which btw is not Postfix but a commercial one, my bad for forgetting
to write that (but the hints did apply!)) was trying to access the
same LDAP directory to verify local usernames).
This is documented. Postfix needs to query the LDAP servers to know if
the domain is local, virtual, relay or none of these. For every
recipient! Thus it makes sense to keep virtual_alias_domains,
relay_domains and mydestination in a file instead of a LDAP server.

Sorry, I was not clear, I meant amavis, not postfix. Postfix is not
configured for LDAP queries at the moment. Amavis was (and was
swamping the directory) in order to get the SpamLover attribute, and
now isn't (since we exported the spamlovers list into a flat file).

Within fwd-rcpt-to section there are no lookups on the amavisd side.
As you stated earlier, it was MTA which was stuck at the recipient
verification for 5 seconds because it was doing a LDAP query,
and Ralf is saying that for a heavily loaded MTA it is advised
to use a local CDB or BDB and load it from LDAP periodically.
This is so regardless of the fact that in your case it was amavisd
which contributed heavily to the LDAP load.

Post by Gianluca Marcari
Amavis was ... in order to get the SpamLover attribute, and
now isn't (since we exported the spamlovers list into a flat file).

Good. Doing the same on the MTA side would be wise too.

Instead of a local file (loaded into a Perl hash at amavisd stratup),
and alternative might be to use a SQLite database to store amavisd-new
recipient attributes. An advantage over a plain file / Perl hash
would be that changing it would not require amavisd reload.
SQLite is a SQL database in a single file (no client/server split).
It is quite fast for mostly-read-only access, making it quite suitable
as a dynamic amavisd lookup backend. It may be worth a try, as
amavisd reloads are rather disruptive events and would better be avoided.

Post by Gianluca Marcari
The odd behaviour is that Amavis, even though it knew our list of 5
local domains, kept querying the directory for every recipient, even
those not local to our systems.

That is intentional. It makes it possible to declare an external recipient
(e.g. a customer) to be a spam lover, making possible to send him any
mail (subject to MTA restrictions on relaying).

Post by Gianluca Marcari
I didn't find a policy bank based on recipient, so I'm looking for
other options in order to change this behaviour.

In principle a solution would be to place a static lookup table
in front of a LDAP lookup table to shunt trivial lookups.
Unfortunately at the moment the placement of LDAP and SQL lookups
is done automatically, they are prepended to the list of each
list of lookups tables in @_maps, so this would need to be made
more flexible. The lookups are general enough, only the construction
of @_maps would need to be made more flexible as far as LDAP and SQL
is concerned.

Mark


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[Tony Earnshaw]

Post by Gianluca Marcari

Post by Ralf Hildebrandt
This is documented. Postfix needs to query the LDAP servers to know if
the domain is local, virtual, relay or none of these. For every
recipient! Thus it makes sense to keep virtual_alias_domains,
relay_domains and mydestination in a file instead of a LDAP server.

Sorry, I was not clear, I meant amavis, not postfix. Postfix is not
configured for LDAP queries at the moment. Amavis was (and was
swamping the directory) in order to get the SpamLover attribute, and
now isn't (since we exported the spamlovers list into a flat file).
The odd behaviour is that Amavis, even though it knew our list of 5
local domains, kept querying the directory for every recipient, even
those not local to our systems.
I didn't find a policy bank based on recipient, so I'm looking for
other options in order to change this behaviour.

1: Ralf had forgotten the Postfix (latest - 2 years) possibility of
proxy_read_maps and what it can do to cut down LDAP server querying. But
then Ralf (probably) doesn't use LDAP;

2: Persuade Mark to put the equivalent of proxy_read_maps into
amavisd-new, the latter which (to be utterly clear, amavisd-new) is, at
the moment, an *enormous* drain on server resources.

Why did you put all your resource-consuming stuff onto Sun machines
(i.e. if you're going to pay brass, why not IBM? Intel or PowerPC - new
stuff for IBM)? Because you happened to have them in stock? Just
curious, I'm an old Sun man (courses, exams and all), converted to a
100% IBM/Red Hat man (experience with all the above).

--Tonni
--
mail: ***@billy.demon.nl
http://www.billy.demon.nl



-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[Gary V]

Post by Gianluca Marcari

Post by Ralf Hildebrandt
This is documented. Postfix needs to query the LDAP servers to know if
the domain is local, virtual, relay or none of these. For every
recipient! Thus it makes sense to keep virtual_alias_domains,
relay_domains and mydestination in a file instead of a LDAP server.

Sorry, I was not clear, I meant amavis, not postfix. Postfix is not
configured for LDAP queries at the moment. Amavis was (and was
swamping the directory) in order to get the SpamLover attribute, and
now isn't (since we exported the spamlovers list into a flat file).
The odd behaviour is that Amavis, even though it knew our list of 5
local domains, kept querying the directory for every recipient, even
those not local to our systems.
I didn't find a policy bank based on recipient, so I'm looking for
other options in order to change this behaviour.
Ciao,
Gianluca Marcari

Matt found that proper indexing helped LDAP queries:
http://marc.theaimsgroup.com/?l=amavis-user&m=112377681617056&w=2

Gary V



-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[Tony Earnshaw]

man, 05.09.2005 kl. 18.03 skrev Gary V:

[...]

Post by Gary V
http://marc.theaimsgroup.com/?l=amavis-user&m=112377681617056&w=2

Indexing (or not) of OpenLDAP attributes will *not* help the number of
connections to the slapd server. It *will* help the speed at which the
server works.

Postfix's proxy_read_maps *will* help the number of connections to the
slapd server.

--Tonni
--
mail: ***@billy.demon.nl
http://www.billy.demon.nl



-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[Clifton Royston]

Post by Tony Earnshaw
[...]

Post by Gary V
http://marc.theaimsgroup.com/?l=amavis-user&m=112377681617056&w=2

Indexing (or not) of OpenLDAP attributes will *not* help the number of
connections to the slapd server. It *will* help the speed at which the
server works.
Postfix's proxy_read_maps *will* help the number of connections to the
slapd server.

If it works anything like proxying does for MySQL, then absolutely.

Proxying queries serves to amortize connection setup and
connection-related resources across many mail processes and queries.
"Proxy" is a bit misleading; while it does proxy the queries, the value
is that it's a connection cache.

I haven't used Postfix proxy_maps with LDAP, but with a MySQL
mailserver setup I did recently, it made the difference between the MTA
completely falling over under moderately high offered load, and the MTA
standing up to everything I could throw of it.

-- Clifton
--
Clifton Royston -- ***@tikitechnologies.com
Tiki Technologies Lead Programmer/Software Architect
"My own personal theory is that this is the very dawn of the world.
We're hardly more than an eyeblink away from the fall of Troy, and
scarcely an interglaciation removed from the Altamira cave painters. We
live in extremely interesting ancient times.
I like this idea. It encourages us to be earnest and ingenious and
brave, as befits ancestral peoples; but keeps us from deciding that
because we don't know all the answers, they must be unknowable and thus
unprofitable to pursue." -- Teresa Nielsen Hayden, 1995


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

[l***@kwsoft.de]

Post by Gianluca Marcari
We are running a few load and stress tests but we'll probably have to
bring 16 more servers in the pool, since our peak load can be 24000
mails per minute.
We will keep tweaking the Amavis configuration because even with 30
servers this will amount to 13 emails/sec sustained and the servers
might not be able to service all requests satisfactorily.

What on earth gets up to 24000 mail/minute incoming and has around 6 million
users as you mentioned before. Is this the gmail.com infrastructure??

Sorry for being curious ..

Andreas




-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/