Support chunk sending in `smtplib`

[jsbucy]

Feature or enhancement

Proposal:

smtplib.SMTP currently has the method data(msg : Union[bytes, str]) to send the message which takes the entire serialized rfc822 message as bytes or str. Email messages can be pretty large -- 10M or more. So it would be nice to be able to send the message in chunks without having to bring it all into the python heap at once. I have a proof-of-concept of this
that adds a parameter data(msg : bytes, last : bool) that I am using in Koukan my mta project.

smtp.data(b'hello\r\n', last=False)
smtp.data(b'world!\r\n', last=True)

I wanted to see if there was any interest in merging this. Building on this, I have another patch to add support for rfc3030 CHUNKING/BDAT which is maybe a separate discussion but if we extend the api here, it ought to be able to support both dotstuff and bdat transparently.

In parallel with this, I am trying to make the corresponding change to aiosmtpd

Has this already been discussed elsewhere?

https://discuss.python.org/t/smtplib-chunked-sending/96804

Links to previous discussion of this feature:

No response

[LamentXU123]

I think its kind of a big FR here. Would you like to start a post on DPO? I think we need wider discussion.

[jsbucy]

Sure thing, thanks! discuss topic

[Zheaoli]

I think RFC3030 should be a good option for us. BTW RFC 3030 gots two parts. BINARYMIME and CHUNKING. I think we just need CHUNKING part right?

cc @python/email-team

[Zheaoli]

@picnixz would you mind help us find a email-team member to review this issue?

[picnixz]

cc @bitdancer

I can also review this but I am not available for the next 10 days for intensive reviews and discussions

[jsbucy]

Thanks all! I'm back from some travel and ready to pick this back up.

I think there's 2 topics here:
1: how to extend the smtplib.SMTP.data() api to not require sending the message body in a contiguous str/bytes. What I have done is kind-of a strawman; there are many possible ways to do it e.g. pass some kind of producer thing.
2: whether to implement BDAT/CHUNKING. This should be able to be transparent to the caller of SMTP.data() and the implementation does dotstuff or bdat per the esmtp capabilities.

Although BDAT is now widely supported, I would like to see a fix for the api that works for dotstuff which will remain common for the foreseeable future. So I have sequenced the changes that way in my patches: do the api change first and then build bdat on top of the extended api.