If you are reading this article, it is probably because you have faced the import/export issue with table fields of type container when using the D365FO Data Management framework. As you have probably noticed, a D365FO data entity container field is by default simply skipped during the export procedure!

Microsoft did not do the best job of documenting the correct approach for solving this issue. It is written only that if an entity has container fields and these fields need to be exported, the entity must implement the getFieldsToBeConvertedToFile() method (see Tips and tricks in the Build and consume data entities tutorial). But other important pieces of information are missing, which is why we decided to explore the import/export logic around D365FO containers by debugging the DMFEntityBase class. We will describe our findings in this article.

Let us set the stage by specifying what D365FO container fields are: they are used for storing binary content like images and documents, and for storing packed objects. An example of binary content is a worker or product image, and examples of packed objects are packed Print management settings and packed Print management condition query.

The two container content types (binary content and packed objects) require different steps for a successful export. We will describe the D365FO data entity container field export rules by using two different examples for these two content types.

Container field with binary content

We will demonstrate how to export the D365FO container field with binary content on a custom DocElectronicSignatureAppearance data entity (Docentric Electronic Signature Appearance). Its main data source is a table named DocElectronicSignatureAppearance that has a SignatureImage field of type container. This field's EDT (DocSignatureImage) is extended from a Bitmap EDT:

Learn more about the Docentric Electronic Signature Appearance data entity >>

To export/import this container field:

1. Add a new string field to the staging table.
2. Give this field the same name as the container field name, with the 'FileName' suffix.
3. Use the FileName EDT for this field.

That’s it! You don’t need anything else for the containers with binary content. When the Data Management framework finds a field with the FileName EDT and suffix during the export, it will take the matching container field and save its content to a separate file. If such a field is found during the import, the DMF framework will know how to identify the matching file and will then save its content to a matching container field. See the Import/Export the package chapter below for instructions on the export/import process.

Container fields with packed objects

Container fields with packed objects (e.g. a packed class or packed query) are a bit more complex. As in the case of a container with binary content, a special field (FileName EDT and suffix) must be added to the staging table AND the famous getFieldsToBeConvertedToFile() method should be implemented on the staging table as well.

We will illustrate this with the Docentric Print Management Settings Package data entity. Its main data source is a standard D365FO Print management settings table (PrintMgmtSettings) that contains 2 container fields: QueryPacked – a print management condition stored as a packed query, and PrintJobSettings - print destination settings stored as a packed object.

The following fields in our data entity are mapped to these containers:

• PrintSettingsQueryPackedV2 data entity field is mapped to PrintMgmtSettings.QueryPacked container field
• PrintJobSettingsV2 data entity field is mapped to PrintMgmtSettings.PrintJobSettings container field

Here are the steps for each of these 2 fields:

1. As in the case of a container with binary content, add one string field to the staging table. The field will have the FileName EDT and the suffix 'FileName' added to the name of the container field. As a result, we see the PrintJobSettingsV2FileName and PrintSettingsQueryPackedV2FileName fields below:
   
   
   
2. Implement the getFieldsToBeConvertedToFile() method on the staging table, where you need to list the container fields added. Here is our implementation:
   
   /// Ssummary: /// When exporting this DMF data entity, PrintJobSettings and PrintSettingsQueryPacked field values will be packaged into separate files. /// Queries are sometimes malformed due to truncation if translated directly into a string. /// /// Returns: A container specifying which fields to extract into a file. public static container getFieldsToBeConvertedToFile() { // Entity fieldname, Staging table field name, StagingFieldName + FileSuffix return [[fieldstr(DocPrintMgmtSettingsPackageEntity, PrintJobSettingsV2), fieldstr(DocPrintMgmtSettingsPackageStaging, PrintJobSettingsV2), fieldstr(DocPrintMgmtSettingsPackageStaging, PrintJobSettingsV2FileName), true], [fieldstr(DocPrintMgmtSettingsPackageEntity, PrintSettingsQueryPackedV2), fieldstr(DocPrintMgmtSettingsPackageStaging, PrintSettingsQueryPackedV2), fieldstr(DocPrintMgmtSettingsPackageStaging, PrintSettingsQueryPackedV2FileName), true]]; }

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

   /// Ssummary: /// When exporting this DMF data entity, PrintJobSettings and PrintSettingsQueryPacked field values will be packaged into separate files. /// Queries are sometimes malformed due to truncation if translated directly into a string. /// /// Returns: A container specifying which fields to extract into a file. public static container getFieldsToBeConvertedToFile() {     // Entity fieldname, Staging table field name, StagingFieldName + FileSuffix     return [[fieldstr(DocPrintMgmtSettingsPackageEntity, PrintJobSettingsV2),             fieldstr(DocPrintMgmtSettingsPackageStaging, PrintJobSettingsV2),             fieldstr(DocPrintMgmtSettingsPackageStaging, PrintJobSettingsV2FileName),             true],             [fieldstr(DocPrintMgmtSettingsPackageEntity, PrintSettingsQueryPackedV2),             fieldstr(DocPrintMgmtSettingsPackageStaging, PrintSettingsQueryPackedV2),             fieldstr(DocPrintMgmtSettingsPackageStaging, PrintSettingsQueryPackedV2FileName),             true]]; }

Import/Export the package

Now when we have prepared our data entities to support the container fields export/import, we are ready to perform the export/import procedure. We will use the Docentric Electronic Signature Appearance data entity in this example.

Export procedure

The contents of the container fields are exported as separate files, one file for each record and each container field equipped for export. Therefore, we want to export into a data package that will contain all these additional files.

After the export is finished, proceed with the download of the package:

The downloaded data package is a .zip file. When you unpack it, you will find the exported XML-Element file (see Docentric Electronic Signature Appearance.xml below) and a Resource folder with all container contents exported as separate files.

The exported XML-Element file contains an XML element with the SignatureImageFileName tag that contains a GUID:

This GUID points to a file in the Resources folder that contains the exported container content:

Import procedure

The images below illustrate the process of importing the package:

After clicking the Upload and add button, we will point to the .zip file created during the export. Once we select the package file (.zip), a data entity is automatically populated based on the information found in the Manifest.xml file from the .zip file.

Note that the import/export procedure for the other data entity used in previous examples, Docentric Print Management Settings Package, is the same.

There is more: Overcome the 32 KB limitation in Data entity Memo field size

When you export the content of a string field with more than 32,768 characters and import that content to a string field, the imported string will be truncated. This is a known limitation according to the Microsoft documentation. If you need to import strings larger than 32,768 characters, Microsoft suggests (without any explanation) that you use container entity fields. Which of the two cases described earlier would be a better fit for this situation? A container with binary content, or a container with packed object? As it turns out, it will be the latter: a combination of an additional field in the staging table and the implementation of the getFieldsToBeConvertedToFile() method on a staging table.

We will demonstrate the necessary steps by using the example of the DocSysEmailMessageTablePackageEntity (Docentric Organization Email Messages Package) data entity. Docentric implemented this data entity to support the import/export of the Organization email templates that are stored in the SysEmailMessageTable table. This table contains two Memo fields: Mail and XSLTMail, which carry the content of the email body.

To support the transfer of the email bodies larger than 32KB, we did the following:

1. We added 2 string fields to the related staging table (DocSysEmailMessageTablePackageStaging), one for the Mail and one for the XSLTMail field. They are of the FileName EDT and have names suffixed with 'FileName':
   
2. We implemented the getFieldsToBeConvertedToFile() method on the staging table where we listed these two fields. Here is the code snippet with our implementation:
   
   /// Summary: /// When exporting this DMF data entity, Mail and XSLTMail will be packaged into a separate files. /// Memo fields are sometimes malformed due to truncation if translated directly into a string. /// /// Returns: A container used to separate the Memo fields into files. public static container getFieldsToBeConvertedToFile() { // Entity field name, Staging table field name, StagingFieldName + FileSuffix return [[fieldstr(DocSysEmailMessageTablePackageEntity, Mail), fieldstr(DocSysEmailMessageTablePackageStaging, Mail), fieldstr(DocSysEmailMessageTablePackageStaging, MailFileName), true], [fieldstr(DocSysEmailMessageTablePackageEntity, XSLTMail), fieldstr(DocSysEmailMessageTablePackageStaging, XSLTMail), fieldstr(DocSysEmailMessageTablePackageStaging, XSLTMailFileName), true]]; }

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

   /// Summary: /// When exporting this DMF data entity, Mail and XSLTMail will be packaged into a separate files. /// Memo fields are sometimes malformed due to truncation if translated directly into a string. /// /// Returns: A container used to separate the Memo fields into files. public static container getFieldsToBeConvertedToFile() {     // Entity field name, Staging table field name, StagingFieldName + FileSuffix     return [[fieldstr(DocSysEmailMessageTablePackageEntity, Mail),             fieldstr(DocSysEmailMessageTablePackageStaging, Mail),             fieldstr(DocSysEmailMessageTablePackageStaging, MailFileName),             true],             [fieldstr(DocSysEmailMessageTablePackageEntity, XSLTMail),             fieldstr(DocSysEmailMessageTablePackageStaging, XSLTMail),             fieldstr(DocSysEmailMessageTablePackageStaging, XSLTMailFileName),             true]]; }

That is all you need to do to export content of Memo fields larger than 32 KB.

Summary

When it comes to exporting and importing the D365FO container fields, we recognize 3 different cases. See our cheat sheet below and remember to export as a package, where each container content will be saved as a separate file.