Explore the internal architecture of every major email format (PST, OST, MBOX, EML, MSG, OLM, Maildir, and PDF/A), binary headers, and direct conversion paths.
On This Page (10 sections) ▾
- 1. The 4 Fundamental Families of Email Formats
- 2. 1. Single-Message Formats
- 3. 2. Monolithic Database and Container Formats
- 4. 3. Directory-Based Server Formats
- 5. 4. Compliance and Archival Formats
- 6. Comprehensive Format Comparison Matrix
- 7. Identifying Email Formats via Magic Bytes
- 8. Common Cross-Format Migration Challenges
- 9. Frequently Asked Questions (FAQs)
- 10. Summary
⚡ Quick Direct Answer: Email File Formats at a Glance
- Outlook (Windows):
.pst(Personal Storage Table) and.ost(Offline Storage Table) — binary B-Tree database files built on Microsoft's NDB layer. - Outlook (Mac):
.olm(Outlook for Mac Archive) — XML-based zip package not directly openable in Windows Outlook. - Standard Open Formats:
.mbox(RFC 4155 Unix standard used by Apple Mail & Thunderbird) and.eml(RFC 822 / 5322 single MIME message). - Single Message Binary:
.msg(Microsoft Compound File Binary / OLE format with discrete MAPI property streams). - Compliance Archiving: PDF/A (ISO 19005 self-contained vector documents preserving headers and text).
Trial Version Notice: The free evaluation versions of Prism email conversion tools export up to 50 items per folder, allowing you to test binary header parsing, MIME stream decoding, and folder tree reconstruction before licensing.
During mail migrations, forensic investigations, and legal eDiscovery, administrators regularly encounter disparate email file formats. Because email clients have evolved distinct storage paradigms over decades, formats range from monolithic B-Tree databases to individual RFC 5322 text files and directory hierarchies.
Simply renaming file extensions (such as changing an .ost file to .pst, or .eml to .msg) will not work because the underlying binary structures, headers, and internal indices differ substantially.
The 4 Fundamental Families of Email Formats
Email storage architectures generally fall into four primary structural categories:
┌────────────────────────────────────────────────────────┐
│ EMAIL FILE FORMAT ECOSYSTEM │
└──────────────────────────┬─────────────────────────────┘
│
┌─────────────────────────┬──────────────────────┴───────────────────┬─────────────────────────┐
│ │ │ │
▼ ▼ ▼ ▼
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
│ SINGLE MESSAGE │ │ MONOLITHIC │ │ DIRECTORY │ │ COMPLIANCE & │
│ FORMATS │ │ CONTAINERS │ │ FILE TREES │ │ ARCHIVING │
├──────────────────┤ ├──────────────────┤ ├──────────────────┤ ├──────────────────┤
│ • EML (RFC 5322) │ │ • PST (Outlook) │ │ • Maildir │ │ • PDF / PDF/A │
│ • MSG (MAPI OLE) │ │ • OST (Exchange) │ │ • Maildir++ │ │ • HTML / MHT │
│ • EMLX (Apple) │ │ • MBOX (Unix) │ │ • MH │ │ • CSV / TXT │
│ • OFT (Template) │ │ • OLM (Mac) │ │ │ │ │
└──────────────────┘ └──────────────────┘ └──────────────────┘ └──────────────────┘1. Single-Message Formats
Individual message files store one email per file, packaging headers, message body, and Base64-encoded attachments into a standalone object.
EML (RFC 822 / RFC 5322)
- Standard: Internet Message Format (RFC 5322) and MIME extensions (RFC 2045–2049).
- Common Clients: Mozilla Thunderbird, Apple Mail, Windows Mail, eM Client, Postfix.
- Internal Structure: Plain text ASCII or UTF-8. Transport headers are positioned at the beginning, separated from the body by a blank CRLF line (
\r\n\r\n). Attachments are stored inline as Base64 MIME parts bounded by boundary delimiters. - Advantages: Human-readable text format, universally indexed by standard filesystem search tools.
- Considerations: Storing hundreds of thousands of individual EML files consumes significant filesystem inode capacity on Linux and Windows file systems.
MSG (Compound File Binary Format)
- Standard: Microsoft Compound File Binary Format (CFBF / OLE Structured Storage).
- Common Clients: Microsoft Outlook for Windows.
- Internal Structure: Acts as a virtual filesystem inside a single file. Specific MAPI properties (such as subject, sender, recipients, and transport headers) reside in discrete binary storage streams identified by hex tags (e.g.,
0x0037forPR_SUBJECT). - Advantages: Retains native Outlook flags, categories, voting buttons, and rich task metadata.
- Considerations: Proprietary format not natively parsed on macOS or Linux without dedicated MAPI libraries or tools such as Prism EML Converter Tool.
EMLX
- Standard: Apple Mail proprietary extension of RFC 822.
- Common Clients: Apple Mail on macOS.
- Internal Structure: Line 1 contains the byte count of the message, followed by the raw RFC 822 MIME payload. The file terminates with an Apple XML property list (
plist) containing macOS Spotlight flags and thread status. Attachments exceeding threshold sizes are stored in companion.emlxpartfiles.
2. Monolithic Database and Container Formats
Monolithic formats package thousands of messages, appointments, contacts, and folder structures into single multi-gigabyte container files.
+-------------------------------------------------------------------------+
| PST / OST DATABASE (NDB LAYER) |
| [Header: Magic '!BDN'] -> [Root B-Tree Index] -> [Node Alloc Table] |
| |--> Inbox Node (Folders & Message Sub-Nodes) |
| |--> Sent Items Node (Encrypted MAPI Descriptors) |
| |--> Attachments Table (Data Blocks & Heap Storage) |
+-------------------------------------------------------------------------+PST (Personal Storage Table)
- Standard: Microsoft Outlook Node Database (NDB) architecture.
- Size Thresholds: 2 GB limit in legacy ANSI format (Outlook 97–2002); 50 GB default limit in modern Unicode format (Outlook 2003+), expandable via administrative registry keys.
- Internal Structure: Multi-level B-Tree database. Uses an allocation table (A4K/AMap) and block index (B-Tree of blocks) to locate MAPI records and heap items.
- Converter Reference: Prism PST Converter Tool.
OST (Offline Storage Table)
- Standard: Outlook Cached Exchange Mode Database.
- Internal Structure: Shares the identical NDB binary layout as a PST file, but binds its encryption keys and folder structures to a specific Exchange Server GUID and user mailbox profile.
- Important Consideration: An orphaned OST cannot be directly mounted or imported into a different Outlook profile. Recovering data requires offline extraction via Prism OST Converter Tool.
MBOX (RFC 4155)
- Standard: RFC 4155. Common variants include
mboxo,mboxrd,mboxcl, andmboxcl2. - Common Clients: Mozilla Thunderbird, Google Takeout, Apple Mail, Alpine.
- Internal Structure: Concatenated flat text file. Every message begins with a mandatory
Fromseparator line and ends with an empty newline. Subfolder trees are represented using companion.sbddirectory structures on the filesystem. - Converter Reference: Prism MBOX Converter Tool.
OLM (Outlook for Mac Archive)
- Standard: Proprietary zip-compressed package used by Microsoft Outlook for Mac.
- Internal Structure: A compressed container storing XML files for message bodies, SQLite/plist indices for folder maps, and raw MIME payload attachments. Windows Outlook cannot read or import OLM files natively.
- Converter Reference: Prism OLM to PST Converter.
3. Directory-Based Server Formats
Directory-based formats store one file per email inside structured filesystem directories, eliminating single-point-of-failure database corruption.
Maildir Directory Structure:
/var/mail/vhosts/domain.com/user/
├── cur/ (Messages that have been opened/read)
├── new/ (Unread incoming messages)
└── tmp/ (In-flight messages currently being written by MTA)- Maildir / Maildir++: Originally designed by Qmail and standard in Dovecot, Postfix, and cPanel environments. The three-directory lifecycle (
tmp/->new/->cur/) prevents race conditions and locking conflicts without requiring database transaction locks.
4. Compliance and Archival Formats
- PDF / PDF/A (ISO 19005): Standardized format for long-term document archiving. PDF/A mandates that all fonts, color profiles, and content streams are fully self-contained. Used extensively in legal discovery and compliance audits to preserve message formatting and header metadata.
- HTML / MHT: Web archive formats that render HTML email bodies and embedded media in standard web browsers without requiring specialized email client software.
Comprehensive Format Comparison Matrix
| Format | Type | Internal Structure | Recommended Limit | Operating System | Header Standard | Outlook Compatibility |
|---|---|---|---|---|---|---|
| .PST | Monolithic DB | Binary B-Tree (NDB) | 50 GB (Unicode) | Windows | MAPI Descriptors | Native Win32 Default |
| .OST | Cached Cache | Encrypted MAPI DB | 50 GB | Windows | MAPI / Profile GUID | Read-only local cache |
| .MBOX | Concatenated Stream | Plaintext From delimiter | 4 GB practical limit | Cross-platform | RFC 5322 MIME | No native direct open |
| .EML | Single Message | Raw Plaintext MIME | Unlimited | Cross-platform | RFC 822 / 5322 | Drag & drop viewing |
| .MSG | Single Message | OLE Structured Binary | Unlimited | Windows | MAPI Properties | Native 1-click open |
| .OLM | Compressed Archive | Zip with XML & MIME | 25 GB | macOS | Custom Microsoft XML | Mac Outlook only |
| .EMLX | Single Message | Byte count + MIME + plist | Unlimited | macOS | RFC 822 + Apple Plist | Apple Mail only |
| .PDF/A | Archival Document | ISO Standard Vector | Unlimited | Cross-platform | Embedded MAPI / MIME | Universal viewer |
Identifying Email Formats via Magic Bytes
Do not rely solely on file extensions when identifying email archives. Inspecting the first few bytes (the "magic bytes") reveals the authentic file type:
# Email Format Magic Byte Identifier
def detect_mail_format(file_path):
with open(file_path, "rb") as f:
header = f.read(16)
if header.startswith(b"!BDN"):
return "Microsoft PST or OST Database (NDB Layer)"
elif header.startswith(b"\xd0\xcf\x11\xe0\xa1\xb1\x1a\xe1"):
return "Microsoft MSG File (OLE Compound Storage)"
elif header.startswith(b"PK\x03\x04"):
return "OLM Archive or Zip Package"
elif header.startswith(b"From "):
return "MBOX Plaintext Concatenated Stream"
elif b"Received:" in header or b"MIME-Version:" in header:
return "EML Plaintext RFC 5322 Message"
else:
return "Unknown or Custom Encrypted Mail Format"Both .pst and .ost files begin with the 4-byte signature !BDN (0x21 0x42 0x44 0x4E). The distinguishing indicator is located at byte offset 0x0A:
- Value
0x0Eor0x0Findicates an open Personal Storage Table (PST). - Value
0x1Bor0x20indicates an Offline Storage Table (OST).
Common Cross-Format Migration Challenges
- MAPI vs. MIME Encoding Translation:
MAPI formats (PST, MSG, OST) store calendar recurrences, task tracking, and rich text within proprietary binary properties. Standard RFC 5322 formats (EML, MBOX) rely on MIME multipart structures. Converting between them requires decoding MAPI properties and unpacking Transport Neutral Encapsulation Format (winmail.dat) streams.
- Hierarchical Representation in Flat Formats:
MBOX files are flat text streams that do not natively record folder hierarchies. Accurate conversion to hierarchical formats requires reading companion .sbd directories or reconstructing folder paths from mailbox metadata.
- Orphaned OST Mailbox Locks:
Because OST files are cryptographically tied to a specific Exchange mailbox GUID, deleting the user account in Microsoft 365 or Active Directory prevents Classic Outlook from opening the local cache. Converting the orphaned OST to a standalone Unicode PST restores access to historical mail.
Frequently Asked Questions (FAQs)
Can I convert MBOX archives directly to PST without losing folder structures?
Yes. Dedicated tools like Prism MBOX Converter Tool read the accompanying .sbd directory hierarchies and reconstruct the exact folder trees within the resulting Unicode PST file.
Why do some EML files contain winmail.dat attachments?
When an email is composed in Outlook using Rich Text Format (RTF), Outlook encapsulates MAPI-specific formatting and attachments into a winmail.dat (TNEF) binary stream. Email clients that do not parse TNEF display this stream as an unreadable attachment.
Which format is recommended for long-term legal and regulatory archiving?
PDF/A (ISO 19005) is widely recommended for legal archives because it locks visual formatting, embeds all necessary font and color data, and remains readable across platforms without dependency on proprietary email software.
Summary
Understanding email file structures is essential for successful mailbox migrations, disaster recovery, and compliance archiving. While plaintext formats like EML and MBOX offer broad cross-platform support, corporate environments often depend on the rich MAPI feature set provided by PST and MSG formats. When transferring mail across incompatible platforms, utilizing specialized tools like Prism PST Converter Tool ensures that metadata, folder hierarchies, and attachments remain intact.
Related Guides:
PST Converter Tool
Export Outlook PST files to multiple formats — PDF, MBOX, EML, Gmail, Office 365, IMAP — without needing Microsoft Outlook installed. Handles large 50GB+ files smoothly. Fast on Windows 11 and Mac (M1–M4).