[milters] Archive

Lists Index Date Thread Search

Article: 982
From: Steve Campbell
Date: 2006-06-13 08:49:09 -0400
Subject: Re: Outbound milter-limit queue files

Removal...........: milters-request@milter.info?subject=remove
More information..: http://www.milter.info/#Support
--------------------------------------------------------

Mr. Howe,

----- Original Message ----- 
From: "Anthony Howe" <achowe@snert.com>
To: <milters@milter.info>
Sent: Tuesday, June 13, 2006 6:04 AM
Subject: [milters] Re: Outbound milter-limit queue files


> Removal...........: milters-request@milter.info?subject=remove
> More information..: http://www.milter.info/#Support
> --------------------------------------------------------
>
> campbell@cnpapers.com wrote:
>>> Steve Campbell wrote:
>>> You probably have in your access.db:
>>>
>>> Connect:127.0.0.1 RELAY
>>
>> I'm fairly certain that, unless the default sendmail MTA config statement 
>> is
>> equivalent to the above, that this is not in the access.db. The server 
>> does feed

Yes, the above _is_ in the access.db, along with a couple of localhost 
entries. Funny how you never notice something until it hits you in the head.

>
> No. Its not the default for sendmail, but pretty much every distribution 
> from Solaris, Linux, BSD, etc. adds this entry.
>
>> mail to the local sendmail server, so somewhere it is allowed. But other 
>> domains
>> can use this as a relay server also. I think I just removed the localhost 
>> or
>> 127.0.0.1 part of the default MTA=localhost or whatever the default RH mc 
>> file uses.
>
> Careful. localhost / 127.0.0.1 is applied in several places in a .mc file, 
> daemon setup, class w, class R, etc.  You just can't delete all references 
> to it.

I was referring to the particular line in sendmail.mc below

DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')dnl

This line is commented out to allow other than localhost to accept mail 
stuff.

>
>>> This is treated a white-list entry in the milter. To do what you want, 
>>> you'll probably need to by-pass the general white-list entry with:
>>>
>>> milter-limit-connect:127.0.0.1 SKIP
>>
>> I doubt that this is "not" working, as I have set myself up to a limit
of 
>> 3/10
>
> Double negative.

I think I meant double negative, I think?
>
>> minutes and sent 3 messages to another of my addresses, and it did follow 
>> the
>> rule by sending the first 2 and waiting about 10 minutes to send the 
>> third. So I
>> have to assume the localhost setting is not coming into play.
>
> When you made the test, did you send from your workstation or from a shell 
> on the same mail host. The latter behaviour is the one at issue.

The mail was submitted through the webmail app on the server itself. My 
workstation only acts as the http client in this case.

>
>> The mail list software, Mailman, is running on the same server as 
>> sendmail. I'm
>
> OK. So you don't have other Sendmail servers feeding the Mailman/Sendmail 
> server.
>
>> also pretty sure it injects messages using SMTP. I'll check tomorrow for 
>> sure,
>
> Then the question becomes what does Mailman do if the local sendmail 
> daemon temp. fails the SMTP session? I don't know if Mailman queue mails 
> the MTA temp. fails. Sendmail won't queue messages (as far as I know) that 
> are temp.failed after the final dot following content. Its the sending 
> client's responsibility.
>
>> but I seem to recall viewing the progress through the logs of the 3 
>> emails
>> mentioned above. Mailman uses a script to pipe the mail into the system 
>> defined
>> in the alias file, and then sends to sendmail.
>
> I'm guessing it invokes sendmail through a inter-process "pipe" and not 
> via LMTP or SMTP, which may be options.

I'll definitely check and see what the Mailman guys can reveal about this 
part.
>
>>> I think to answer better requires further clarification of how your mail 
>>> system is setup and the flow of mail through it. Do you have several 
>>> systems feeding the list server? Or is all-in-one system. The former 
>>> should be easy to handle, while the latter is unclear.
>>
>> The server is a generic sendmail 8.12 Tao server. It's main purpose is to 
>> handle
>
> You haven't updated to 8.13.6 yet? Naughty.

Not yet. I like to keep the distro clean, but downloaded the 8.13 rpms from 
another unrelated site some time ago. Maybe now is a good time to try them. 
(Maybe not - the simple things are what bites you sometimes.)
>
>> the IMAP Horde/Imp webmail stuff, but also acts as a tertiary MX for a 
>> few
>> domains. It's load is very light, so I stuck Mailman on it as a 
>> replacement for
>> 12-All, which is not performing as promised. I used the IMP stuff to send 
>> the 3
>
> OK. web server > sendmail > mailman > sendmail > internet

Correct.
>
>> emails to test with. Mailman might throw a curve with those alias 
>> scripts, but I
>> don't think it will based on what I've seen. The Mailman config option is 
>> an
>> SMTP based option.
>
> Hmm. I thinking if the mail is inserted via a inter-process pipe or SMTP 
> on the localhost, I suspect no queuing will happen and it would be 
> Mailman's job. If it comes in via LMTP, that is queued by definition 
> before being passed to the SMTP daemon, which should be limited as you 
> would expect.
>
>> I plan on making a list server that just runs Mailman, as an end result. 
>> The
>> purpose of milter-limit is to throttle the output so as not to suck up 
>> all of
>> the bandwidth I have as it pumps out the thousands of emails (all to 
>> subscribed
>> recipients - I'm not a spammer).
>
> I keep thinking there is a better way than using milter-limit for this 
> case using sendmail queue management configuration options.
>
> What you want probably is to have a dedicated sendmail queue for Mailman 
> where all mail from Mailman is first queued and they processed in chunks 
> at regular intervals.
>
> QUEUE_GROUP(
> `slowmail', `Path=/var/spool/mqueue/slowmail,
> Interval=15m, Jobs=200, Nice=10'
> )
>
> There are more options for QUEUE_GROUP, so you'll need to review sendmail 
> documentation.
>
> You'll probably need a ruleset too to place mail *sent* by Mailman into 
> slowmail queue:
>
> LOCAL_RULESETS
> Squeuegroup
> R $* TAB $: $>canonify $&f
> R mailman.user <@ list.domain > TAB $# slowmail
>
> Be sure to replace TAB by a real tab character. You can have multiple 
> queue groups and modify the ruleset to queue to different queues by 
> recipient or sender; see Bat Book 3e chapter 11.
>
> Other related options, probably used for the default mqueue:
>
> define(`confMAX_RCPTS_PER_MESSAGE', limit)
>
> dnl Affects ALL queues.
> define(`confDELIVERY_MODE', `deferred')
> define(`confDELIVERY_MODE', `queueonly')
>
> define(`confMAX_QUEUE_RUN_SIZE', limit);
>
> See the sendmail cf/README and/or doc/op.txt or the online versions found 
> at sendmail.org for details or see the Bat Book 3e.
>
> I've never had need for separate queues, but I think it probably the 
> solution you want. The above is digested highlights of chapter 11.

I've got the Bat Book 2e, so I'll need to upgrade something else now, also. 
It doesn't have the queue stuff in it, mostly due to it highlighting before 
8.12.
>
>> Anyway, this is more of a "what does sendmail do when it receives it's 
>> own
>> temporary failure" (from milter-limit) type of a question. I don't think 
>> I ever
>> considered a situation like this, but know there are some options in 
>> sendmail to
>
> Between two sendmail processes, the files would be queued by the sender. 
> Internally by a sendmail process that first receives the message is 
> untried.
>
> Having read ch 11 on queues, you might be best to go that route (remember 
> to remove the milter-limit-connect:127... entry in access.db).
>
>> do something like this. The obvious thing would be put the emails in the 
>> output
>> queue and wait until the condition clears,  to send out the remaining 
>> emails,
>> but this didn't seem to be the case. I'm now wondering, though, where
>> milter-limit takes over the transaction. And will Mailman see these as 
>> real
>> failures.
>
> milter-limit doesn't handle the transaction directly. Its consulted. It 
> can temp.fail, reject, disard at the MAIL FROM:, RCPT TO: or tag, 
> quarantine after the message is received.
>
>> I should probably set up a little more complex test, and see how it goes. 
>> Maybe
>> the simple stuff wasn't hard enough for sendmail, Mailman, or 
>> milter-limit,
>> although I don't see how it would matter. If you have anything to add, I
>> certainly respect your knowledge of the matter. I'll also throw this at 
>> the
>> Mailman list to see if they can shed some light on what might occur.

After catching up on all of the great suggestions and questions needing 
answers you have provided me, I hope you don't mind me reopening this thread 
to either define my solutions or asking more questions.

You have been a great help, and sendmail always seems to have more to learn 
about no matter what the situation.

Thanks very much.

Steve
>
> -- 
> Anthony C Howe          Skype: SirWumpus                    SnertSoft
> +33 6 11 89 73 78         AIM: SirWumpus    Sendmail Milter Solutions
> http://www.snert.com/     ICQ:
7116561      http://www.snertsoft.com/
> 



Lists Index Date Thread Search