Problem/Motivation

Multipart (MIME entities) messages can have mail attachments. As a part of #2804543: [META] Provide a complex Inmail element issue, we want to be able to display mail attachments as well.

Proposed resolution

Similarly to #2804545: Display email's inline images using new element, we should create a new element (and a twig template) that handles "attachment" case. We would need to check "attachment" in content disposition header and act in these cases. See: https://tools.ietf.org/html/rfc2183

Regarding the display, we could just display the file name and it's extension. Also, the great UX would be to have this clickable. In that case the attachment would open/download.

Remaining tasks

User interface changes

API changes

Data model changes

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

mbovan created an issue. See original summary.

miro_dietiker’s picture

Priority: Normal » Critical
Related issues: +#2805533: META Special treatment for attachment / image parts in parser

Without this, mails might be huge, eating large amount of space in DB or filesystem, without any value at all.

One of the key points with this code is, we should check the resource / memory usage and try to be very friendly.
Inmail could introduce the idea to offload attachments to temporary files when parsing, or skip them at all until they are really accessed.

See the API / Parser level issue: #2805533: META Special treatment for attachment / image parts in parser

miro_dietiker’s picture

miro_dietiker’s picture

mbovan’s picture

Status: Active » Needs review
FileSize
11.08 KB

Created a patch based on inputs above and discussions in #2804587: Create mail display mock-ups, #2807719: Readd the body for multipart messages.

Also, providing simple tests based on #2807733: Write mail display tests per example.

I think there is space to improve parsing logic, but my concern is whether should we do it in prerendering or during the core Inmail parsing phase?
That said, we could think about introducing "Attachment" (Inmail) entity as there are many things describing it: filename, content type, attachment type (image, video, document), encoding, raw content, offloaded temporary file (entity)? etc... That would certainly help us to avoid complex prerendering logic and to parse an email much better.

miro_dietiker’s picture

It would not be the parser that is extended. The parser just identifies the mail standard elements and does this correct following the standard. The missing piece there is adress parsing.

We could introduce a new helper that converts / filters / accesses these elements to be more prepared for display.

But for now i would recommend put this into preprocess and move the logic into a helper with follow-ups. I highly recommend to make the features just work as simple as possible with the first iteration (incl test coverage).

mbovan’s picture

Moved build-attachments logic into preprocess hook.

Created a follow-up for attachment identifier/helper method.

miro_dietiker’s picture

Would be really nice to get a status screenshot if you implement some feature like this with visual elements. It takes quite some steps to setup an environment to test drive it and "see" it to understand if it works for the user or not.

miro_dietiker’s picture

Status: Needs review » Needs work
  1. +++ b/src/Tests/InmailMessageWebTest.php
    @@ -0,0 +1,64 @@
    +    $this->assertRaw('<img src="" alt="hello.txt" class="inmail-message--attachment-text">');
    ...
    +    $this->assertRaw('<img src="" alt="square.png" class="inmail-message--attachment-image">');
    

    Why an image without source and figcaption?
    What we semantically deliver is more a bullet item than an image with metadata.

  2. +++ b/src/Tests/InmailMessageWebTest.php
    @@ -0,0 +1,64 @@
    +    $this->assertRaw('<figcaption>hello.txt</figcaption>');
    ...
    +    $this->assertRaw('<figcaption>square.png</figcaption>');
    

    I'm missing a basic file size indication.

  3. +++ b/tests/modules/inmail_test/eml/attachments/simple.eml
    @@ -0,0 +1,40 @@
    +Content-Type: text/plain; charset=US-ASCII; name="hello.txt"
    +Content-Disposition: attachment; filename="hello.txt"
    ...
    +Content-Type: image/png; name="square.png"
    +Content-Disposition: attachment; filename="square.png"
    

    Please test with longer filenames and also special chars.

    And also add other examples such as Content-Disposition: inline.

mbovan’s picture

FileSize
53.15 KB

Re #8:
Ah, sorry, here is the screenshot. I thought there is not too much to see except the list of items:

Re #9:

Why an image without source and figcaption?
What we semantically deliver is more a bullet item than an image with metadata.

No source as we don't have mockups/images yet. There is a comment in Twig to add it when it's ready.
Yes, then we can remove figure/figcaption and just use <img> in <li>.

I'm missing a basic file size indication.

Skipped this part as I thought we'll use the correct file size when we have attachments offloading to temporary files.
I'll provide some rough estimation based on an encoded string.

Please test with longer filenames and also special chars.
And also add other examples such as Content-Disposition: inline.

Inline (images) are currently not supported. I am going to extend the logic to support them in this issue as well.

mbovan’s picture

Addressed #9.

Added a file size indicator, added support for inline images (as attachments) and removed <figcaption>.

Improved tests with several attachments of images, text files, special characters in file names, long file names etc.
Because of this, the patch is going to be ~100KB big. Also, had some issues to assert the special characters in the file name, so the patch is currently asserting "non-special" part and the file size only.

Here is the current screenshot:

We can probably make all the helper methods cleaner in #2817007: Introduce attachment identifier helper...

Edit: Related issue update.

miro_dietiker’s picture

Status: Needs review » Needs work

Cool.

But we don't commit large files just to verify file size asserts. In that case, i trust the code works.
If you really need to pass in large data into the mail, you can pick any file or create dummy data and do base64 encode.

Follow-Up: The special characters will be important and need to be asserted.. Note though that the mail body is in different encoding than the browser, so you need to think properly about both collation and encoding. You might need to inline-base64-encode the filename header...

+++ b/inmail.module
@@ -351,7 +355,90 @@ function inmail_theme() {
+function inmail_preprocess_inmail_multipart_message(&$variables) {
...
+    foreach (InmailMessage::filterMessageParts($message) as $type => $message_parts) {
...
+            $variables['attachments'][] = inmail_message_build_attachment($message_part);
...
+          $variables['unknown'] = [];

+++ b/src/Element/InmailMessage.php
@@ -69,37 +66,48 @@ class InmailMessage extends RenderElement {
+  public static function filterMessageParts(MultipartEntity $multipart_entity) {
...
     foreach ($multipart_entity->getParts() as $message_part) {
...
+            $elements['attachments'][] = $message_part;
...
+            $elements['inline'][] = $message_part;
...
+            $elements['unknown'][] = $message_part;

You should start to track which parts (numeric keys) are used so later code knows about it. Don't drop the numbers because it will no more be possible to return a reference to it. For instance (follow-up), a download filename will need to link to the part for download, which will need the part key.

mbovan’s picture

Yeah, removed the big files and replaced them smaller ones (~1KB).

Created followups. Also, index reference path seems much more complicated than I thought at first, so I created a followup for it.

In this patch, I added $index which is (the first level) message part index. That works fine for attachments since they are always "attached" to the main Multipart message, while there are issues with sub-elements (related/alternative which is not supported for display yet) and inline images. When we start with #2817007: Introduce attachment identifier helper these issues might be fixed by themselves though...

miro_dietiker’s picture

Status: Needs review » Fixed

Committed. More stuff, display adjustment in follow-ups.

+++ b/inmail.module
@@ -351,7 +355,90 @@ function inmail_theme() {
+function inmail_message_get_attachment_file_size($content, $encoding, $precision = 2) {
...
+  $suffixes = ['B', 'KB', 'MB', 'GB', 'TB'];
...
+  return round(pow(1024, $base - floor($base)), $precision) . ' ' . $suffixes[floor($base)];

Follow-Up: This would be the perfect example for a unit test.

miro_dietiker’s picture

Oops, PHPStorm also warns about this piece:
$suffixes[floor($base)]

floor() still returns float and is not save as array key. You need to cast it to (int).

miro_dietiker’s picture

Status: Fixed » Needs work

Tested a bit more, did some longer review, testing multipart worked most easily with this:
cat tests/modules/inmail_test/eml/attachments/complex.eml | drush --uri=http://d8.dev/ inmail-process drush

Follow-Up: The headers still don't look like the mockup.
This also applies to the attachment list. That list should also have a semantical title "Attachments".
We should start with some wrappers and reduce the header area. It really needs to start look nice.

Follow-Up: If i accidentally forget to provide the deliverer, i get these error messages:

 [warning] array_flip(): Can only flip STRING and INTEGER values! EntityStorageBase.php:227
 [error]  Deliverer "" not found 

There is no feedback on the terminal about processing.
Follow-Up:
\Drupal\inmail\MessageProcessor::process should return the ProcessorResult... and later processMultiple should return an array of these by preserving the keys.
Then, drush_inmail_process() can check for verbose flag and output feedback.
Also from the Paste email tab, the response is just "The message has been processed.". It should additionally display a link to the log record if logging is enabled.

Follow-Up: Improve logging
The log record created displays two valueless arguments:
DRUPAL\INMAIL\DEFAULTANALYZERRESULT
[contexts] (object):
MODERATORFORWARDHANDLER
[0] (object):
[string] (string): Moderator email address not set
This seems pretty useless to me. There is an issue to build a meaningful context summary for logging.
Funnily, the ForwardHandler doesn't report anything if it forwards the mail. Instead it logs if it is unconfigured... Do we need log severities?

Follow-Up: Download attachments on click
This will need a passthrough route. The element doesn't know where an element is stored. So the element should get an attachment route key from the caller. If not provided, the attachments are not linked.

Follow-Up: Deliver images
First simply deliver them so that they are displayed in a browser. Later also support inline images.

Back to needs-work to create the follow-up issues. Please close when created.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.