This article is now available in our new knowledge base: Send files larger than 2 GB to AWS from IBM i
Zend Server for IBM i is a 32 bit program, which means it is limited to file sizes around 2 GB. This example shows how to use the split command in PASE to break the file into smaller chunks and send them up to AWS. The split command is a standard command in Linux, so this technique would also apply there.
Any version of Zend Server for IBM i.
The way to accomplish this with 32 bit PHP is to 'chunk' the file. To 'chunk' the file means to break it up into smaller pieces, and transmit the pieces. PASE uses the 'split' command to break up the file. Then the AWS SDK provides for sending the pieces. You can read the AWS documentation here:
Chunking the file is fairly straightforward. Here is the format for the PASE command:
$ split -b 1000m /path/to/large/file /path/to/output/file/prefix
You end up with files having prefix_aa, prefix_ab, so you can then pass each chunk to uploadPart, and finally call completeUpload and you’re done. Here is an example PASE session:
$ split -b 500m test testnew
$ ls -la testnew*
-rw-r--r-- 1 maurice 0 524288000 Mar 10 14:38 testnewaa
-rw-r--r-- 1 maurice 0 524288000 Mar 10 14:39 testnewab
-rw-r--r-- 1 maurice 0 524288000 Mar 10 14:39 testnewac
-rw-r--r-- 1 maurice 0 524288000 Mar 10 14:40 testnewad
-rw-r--r-- 1 maurice 0 314572800 Mar 10 14:40 testnewae
So you call
Aws\S3\S3Client::createMultipartUpload() which will initialize the upload and give you an upload id.
Aws\S3\S3Client::uploadPart()with a chunk and a part number, iteratively or in parallel for all the chunks.
Finally you call
Aws\S3\S3Client::completeMultipartUpload() to finalize the upload.
For code examples and more information, please refer to the AWS documentation in the above link.