mxHERO Cloud Storage Load Balancing Guide
Table of Contents
How mxHERO manages storage accounts when using Protect Attachment
This guide explains how mxHERO manages cloud storage usage when processing email attachments with features such as Protect Attachment. It describes how mxHERO ensures reliability, scalability, and optimal performance while respecting cloud storage provider rate limits.
1. Email Processing Model in mxHERO
mxHERO processes email at the recipient level, not only at the message level. This means that:
- Each email is processed independently for each recipient.
- A single message sent to multiple recipients results in multiple independent processing operations.
Example:
- Message ID: MSG-001
- Recipients: 10
- Result: 10 independent processing operations
Each processing operation evaluates the message, applies the configured rules, and performs storage operations if required.
This architecture allows precise control but requires careful coordination when interacting with cloud storage providers.
2. Why Load Balancing Is Necessary
Cloud storage providers (such as Box, OneDrive, or Google Drive) enforce strict rate limits, including:
- Maximum concurrent uploads
- API request limits per second
- Throughput limitations per account
If too many uploads are attempted simultaneously, the provider may throttle or reject requests.
Example:
- One email sent to 1,000 recipients
- One storage account configured
- Without concurrency control, 1,000 simultaneous upload attempts would occur
- This would saturate the storage account and cause throttling or failures
To prevent this, mxHERO implements its own internal load balancing and rate control mechanisms.
3. mxHERO Internal Rate Limiting and Queueing
mxHERO protects your storage accounts by enforcing controlled access.
When storage operations are required:
- mxHERO allows only a safe number of concurrent uploads per storage account.
- Additional processing operations are placed in an internal queue.
- Queued operations wait until storage capacity becomes available.
- As active uploads complete, queued operations resume automatically.
This ensures:
- Storage provider rate limits are respected
- Upload operations remain stable and reliable
- All recipients are eventually processed successfully
This queueing mechanism is internal to mxHERO and fully automatic.
4. How mxHERO Uses Multiple Storage Accounts
When multiple storage accounts are configured in a rule, mxHERO does not use a strict round-robin selection model. Instead, mxHERO uses an availability-based selection approach:
- mxHERO selects storage accounts dynamically based on availability.
- Storage accounts may be selected in a randomized or non-sequential manner.
- When a storage account reaches its safe concurrency limit, mxHERO temporarily stops assigning new uploads to that account.
- mxHERO continues assigning uploads to other storage accounts that remain available.
- As saturated accounts become available again, they are automatically reintroduced into the pool.
This approach ensures:
- Maximum throughput
- Automatic load distribution
- Avoidance of storage account saturation
- Optimal use of all configured storage resources
This dynamic balancing provides better performance than fixed round-robin assignment, especially under high load.
5. Recommended Best Practice: Use Shared Links When Possible
mxHERO provides an option to use the same storage link for all recipients of a message. This is the recommended configuration whenever possible.
How Shared Link Mode Works
Even though mxHERO still processes each recipient independently:
- Only the first processing operation uploads the attachment to cloud storage.
- Other processing operations are temporarily queued.
- Once the upload completes, mxHERO stores the resulting storage link in its internal cache.
- All remaining recipients reuse the same cached link.
- No additional uploads are performed.
6. Benefits of Using Shared Links
Using shared links provides significant advantages:
- Reduced Storage Load
- Only one upload is performed per message, regardless of recipient count.
- Faster Processing
- Subsequent recipients do not require storage uploads.
- Improved Scalability
- Handles very large recipient lists efficiently.
- Reduced Risk of Rate Limiting
- Minimizes storage API usage.
- Optimal Performance
- Reduces queue times and improves overall throughput.
For these reasons, mxHERO strongly recommends using the shared link option whenever business and security requirements allow it.
7. Processing Example Comparison
Without Shared Links
Email with 500 recipients:
- Up to 500 storage uploads required
- mxHERO balances uploads across available accounts
- Internal queue controls concurrency
With Shared Links (Recommended)
Email with 500 recipients:
- Only 1 storage upload required
- Remaining recipients reuse cached link
- Minimal storage load
- Maximum efficiency
8. Summary
mxHERO uses intelligent internal load balancing to protect storage accounts and ensure reliable processing.
Key points:
- Emails are processed independently per recipient.
- Storage uploads are controlled by internal concurrency limits.
- mxHERO dynamically selects available storage accounts rather than using strict round-robin assignment.
- Internal queueing ensures stability and reliability.
- Using shared links is strongly recommended to maximize performance and efficiency.
All of these mechanisms operate automatically to ensure secure, scalable, and reliable attachment protection."