The Brightspace API employs two general methods for uploading binary data:
Simple upload. API actions that use the simple (also called “combined”) upload mechanism assume that the data passed in from the client will be of a small to medium size (for example, a user’s profile image).
Resumable upload. API actions that use the resumable upload mechanism assume that the data passed in from the client could be of considerable size (for example, large photo images or video data destined for ePortfolio) and perhaps along a less reliable communication channel (for example, a cellular network), and thus provide a mechanism for uploading the data in chunks in a resumable structure.
Basic concepts¶
Uploaded data and LMS item are two different things. Every item in the Learning Framework that’s backed by uploaded file data (an image, a video, a document) binds together two conceptual bits of information: the LMS object that represents the data in the Learning Framework (an eP Artifact, a Dropbox Submission Attachment), and the binary data that the user or client application provides for that object (the image or video or document data uploaded). It’s helpful to be conscious of this difference to grasp how the upload process works.
Authentication controls around the object ID. Because the binary data uploaded is essentially transitory until it gets bound into the Learning Framework, the LMS service focuses authentication controls around the acquiring and use of the upload key, and on the process of uploading the data itself. If a client can’t successfully attach the uploaded binary data to an LMS object, then eventually the service will purge the uploaded data from its temporary storage.
Authentication tokens on the request. In the HTTP request examples that follow, we have left out the mechanisms used for authenticating the request. You should consider that the authentication information will appear in your request, and be different depending on the authentication method you are using for your calls.
If your client uses the OAuth2 authentication system, then the HTTP request will include your authentication bearer token in the Authentication header of the request, like in this example of the same call:
PUT https://someLMShost.edu/d2l/api/lr/{version}/objects/?repositoryId={repo_ID_parm} HTTP/1.1 Authentication: Bearer eyJ0eXAiOiJKV1Q...{rest of bearer token} Content-Type: multipart/form-data; boundary=xxBOUNDARYxx Content-Length: {PUT body length in bytes}
If your client uses the legacy ID/Key-based authentication system, then the HTTP request URL will have your authentication tokens included as query parameters on the request URL, like the x_ parameters in this example:
PUT https://someLMShost.edu/d2l/api/lr/{version}/objects/?x_t={timestamp}&x_a={app_id}&x_b={user_id}&x_d={user_sig}&x_c={app_sig}&repositoryId={repo_ID_parm} HTTP/1.1 Content-Type: multipart/form-data; boundary=xxBOUNDARYxx Content-Length: {PUT body length in bytes}
Simple uploads¶
Simple (or “combined”) upload APIs support uploading data of a reasonably small size. These actions expect the client to provide all the file data with a single action.
Note
The size limit for simple uploads is set when the back-end Learning Service gets deployed, and is generally set at approximately 488 MB. It is not an exact value you can assume, nor is it discoverable through the Brightspace APIs. You can seek to confirm the exactly value for a particular service with the service administrator, but if your application needs to accommodate a variety of Learning Services, you should assume that any upload with a body larger than 488MB may exceed this limit.
The size limit applies to the entire API call body, and not just each individual file you’re uploading.
The exact way that simple uploads get done varies with the product component, but there are two general types:
- RFC1867 HTTP file upload
Profile image and LR package uploading gets done using the simple POST form file upload described in RFC 1867.
Upload to Learning Repository. Here’s an example showing what the actual HTTP request looks like for a simple upload of a new Learning Repository package:
PUT https://someLMShost.edu/d2l/api/lr/{version}/objects/?repositoryId={repo_ID_parm} HTTP/1.1 Content-Type: multipart/form-data; boundary=xxBOUNDARYxx Content-Length: {PUT body length in bytes} --xxBOUNDARYxx Content-Disposition: form-data; name="Resource"; filename="TEST_SCORM.zip" Content-Type: application/zip {file data bytes} --xxBOUNDARYxx--Updating the profile image. Here’s an example showing what the actual HTTP request looks like for a user updating their own profile image:
POST https://someLMShost.edu/d2l/api/lp/{version}/profile/myProfile/image HTTP/1.1 Content-Type: multipart/form-data; boundary=xxBOUNDARYxx Content-Length: {POST body length in bytes} --xxBOUNDARYxx Content-Disposition: form-data; name="profileImage"; filename="my_new_profile_image.png" Content-Type: image/png {image file data bytes} --xxBOUNDARYxx--Upload to create a new course import job. Her’s an example showing what the actual HTTP request looks like for
creating a new course import job request
:POST https://someLMShost.edu/d2l/api/le/{version}/import/{org-unit-id}/imports/ HTTP/1.1 Content-Type: multipart/form-data; boundary=xxBOUNDARYxx Content-Length: {POST body length in bytes} --xxBOUNDARYxx Content-Disposition: form-data; name="file"; filename="ImportPackage.zip" Content-Type: application/zip {import package data bytes} --xxBOUNDARYxx--Upload to ePortfolio. Here are two examples showing what uploads to ePortfolio look like.
This shows a simple artifact file upload:
POST https://someLMShost.edu/d2l/api/eP/{version}/artifacts/file/upload HTTP/1.1 Content-Type: multipart/form-data; boundary=xxBOUNDARYxx Content-Length: {POST body length in bytes} --xxBOUNDARYxx Content-Type: text/plain Content-Disposition: form-data; name="name" UploadFileName.txt --xxBOUNDARYxx Content-Type: text/plain Content-Disposition: form-data; name="description" Description for the uploaded file. --xxBOUNDARYxx Content-Type: text/plain Content-Disposition: form-data; name="file"; filename="UploadFileName.txt" {file data bytes} --xxBOUNDARYxx--This shows an eP package import to multiple users:
POST https://someLMShost.edu/d2l/api/eP/{version}/import/new HTTP/1.1 Content-Type: multipart/form-data; boundary=xxBOUNDARYxx Content-Length: {POST body length in bytes} --xxBOUNDARYxx Content-Type: text/plain Content-Disposition: form-data; name="targetUsers" 12345 --xxBOUNDARYxx Content-Type: text/plain Content-Disposition: form-data; name="targetUsers" 67890 --xxBOUNDARYxx Content-Type: applicaiton/zip Content-Disposition: form-data; name="file"; filename="import-package.zip" {file data bytes} --xxBOUNDARYxxSome items of note:
You must provide a top-level Content-Length header that declares the length of the entire request body in bytes.
The Content-Disposition header for the part you’re uploading will have a name field whose value varies from API call to API call; in the LR case above, it must be “Resource”; in the case of uploading an attachment for a news item, it should be “file”. Providing the wrong value here can mean that the back-end service will not be able to properly hand off the uploaded data the service tool using it.
- RFC2388 Multipart/mixed
Medium-sized simple file uploading (for example, dropbox submissions, or creating news events or discussion posts with attachments) gets done using a multipart/mixed method similar to that described in RFC 2388. These actions typically use a JSON structure as the first part, to provide the meta-data for the file to upload, followed by a part containing the file’s actual data.
Upload to course content. Here’s an example showing what the actual HTTP request looks like for a file topic upload to course content:
POST https://someLMShost.edu/d2l/api/le/{version}/{orgUnit}/content/modules/{moduleId}/structure/ HTTP/1.1 Content-Type: multipart/mixed;boundary=xxBOUNDARYxx Content-Length: {POST body in length in bytes} --xxBOUNDARYxx Content-Type: application/json { "TopicType": 1, "Url": "/content/enforced/170315-EXT-101/test_html_file.html", "StartDate": null, "EndDate": null, "DueDate": null, "IsHidden": false, "IsLocked": false, "Title": "TestTitle", "ShortTitle": "Test", "Type": 1, "Description": {"Content": "This is a test file", "Type": "Text"}, "MajorUpdate": null, "MajorUpdateText": null, "ResetCompletionTracking": null } --xxBOUNDARYxx Content-Disposition: form-data; name=""; filename="file.txt" Content-Type: text/plain This is a sample text file with some text content. --xxBOUNDARYxx--Upload to dropbox. Here’s an example showing what the actual HTTP request looks like for a simple upload to a dropbox folder:
POST https://someLMShost.edu/d2l/api/le/{version}/{orgUnit}/dropbox/folders/{folderId}/submissions/mysubmissions HTTP/1.1 Content-Type: multipart/mixed; boundary=xxBOUNDARYxx Content-Length: {POST body length in bytes} --xxBOUNDARYxx Content-Type: application/json { "Text": "Here you go", "Html": null } --xxBOUNDARYxx Content-Disposition: form-data; name=""; filename="testFile.jpg" Content-Type: image/jpeg {binary JPEG data from file} --xxBOUNDARYxx--Upload to news. Here’s an example showing what the actual HTTP request looks like for a news event with two attachments (a discussion with attachments uses a similar structure):
POST https://someLMShost.edu/d2l/api/le/{version}/{orgUnit}/news/ HTTP/1.1 Content-Type: multipart/mixed;boundary=xxBOUNDARYxx Content-Length: {POST body length in bytes} --xxBOUNDARYxx Content-Type: application/json { "EndDate": null, "IsPublished": true, "ShowOnlyInCourseOfferings": false, "Title": "Test title", "Body": { "Text": "Test body text", "Html": null }, "StartDate": "2013-02-20T13:15:30.067Z", "IsGlobal": false } --xxBOUNDARYxx Content-Disposition: form-data; name="file 0"; filename="file.txt" Content-Type: text/plain This is a sample text file with some text content. --xxBOUNDARYxx Content-Disposition: form-data; name="file 1"; filename="img-225x225.png" Content-Type: image/png {image data here} --xxBOUNDARYxx--Notes.
You must provide a top-level Content-Length header that declares the length of the entire request body in bytes.
The first part of the multipart upload request must be the JSON block.
The JSON block’s precise contents might vary from request to request: please double-check that you’re using the right structure, depending upon the upload route you’re using.
The previous example shows RichText JSON block used by the dropbox folder submissions route.
If you’re uploading a content file topic to a course offering’s content store, then you should instead use a
Content.ContentObjectData
JSON block.The Content-Disposition header for the file data you’re uploading should include an empty name field, and a filename field with the file name for the file you’re uploading on the local file system.
Resumable uploads¶
The protocol for a resumable upload comprises three overall steps, summarized here and more fully explained further on in this section:
The client must initiate a new upload via an “upload” action to acquire a unique upload key from the service.
The client then uses this upload key to upload the binary data to a temporary location on the service.
Finally, the client must use a “save” action, providing the upload key, to direct the service to attach the temporary file data to an LMS object in the service.
Note that the precise actions used for resumable uploads may vary from API domain to domain, but the general steps will be the same, illustrated in the following diagram.
Note
The size limit for each single chunk of data is set when the back-end Learning Service gets deployed, and is generally set at approximately 488 MB. It is not an exact value you can assume, nor is it discoverable through the Brightspace APIs. You can seek to confirm the exactly value for a particular service with the service administrator, but if your application needs to accommodate a variety of Learning Services, you should assume that any upload with a chunk larger than 488MB may exceed this limit.
sequenceDiagram autonumber participant Client participant Service note over Client,Service: Acquire upload key Client->>Service: {new upload action}<br/>X-Upload-Content-Type: {mime-type}<br/>X-Upload-Content-Length: {total-zie}<br/>X-Upload-File-Name: {file-name} Service-->>Client: 308 Resume Incomplete<br/>Location: /d2l/upload/{key} note over Client,Service: Upload file data Client->>Service: POST https://{service.domain}/d2l/upload/{key}<br/>Content-Type: {mime-type}<br/>Content-Range: bytes {byte-range-A}/{total-size} Service-->>Client: 308 Resume Incomplete<br/>Location: /d2l/upload/{key}<br/>Range: {byte-range-A} Client->>Service: POST https://{service.domain}/d2l/upload/{key}<br/>Content-Type: {mime-type}<br/>Content-Range: bytes {byte-range-B}/{total-size} Service-->>Client: 200 OK note over Client,Service: Attach upload to the LMS Client->>Service: POST {attach file action}<br/>UploadKey={key} Service-->>Client: 200 OK
Acquire an upload key¶
The client will not know, nor should it care, where precisely an uploaded file
will go on the service (nor what key or ID will get associated with the
file). Accordingly, in the first stage (action 1 in the diagram), the client
application should notify the service that it intends to initiate a resumable
upload process – it does so by using one of the new upload actions available
in the API. The various product components may provide slightly different routes
for this purpose: for example, ePortfolio provides a
new file upload
action for this purpose, while Learning Environment provides an
upload
action.
Within the new upload action, the client must provide three special HTTP headers:
- X-Upload-Content-Type: {mime-type}
The client should provide the mime-type for the data it wants to upload.
- X-Upload-Content-Length: {total-size}
When requesting the upload key, the client should indicate the file’s total size, in bytes.
- X-Upload-File-Name: {file-name}
The client should provide the file name of the file it will upload.
If the calling user context has permission to submit files within the particular LMS product component context supporting the action it’s calling, the server will respond with a 308 Resume Incomplete response containing a special HTTP header:
- Location: {upload path}
The server provides the client with the location URL it should use to upload the file data. The last component in this upload path will be the upload key, and the client should make a separate record of this token, as it will need to use this key later on in the process (to instruct the server to attach the uploaded data to the LMS).
Note
Because the first step ends with the back-end service sending a 308 status code, some client libraries may treat that as a redirection and automatically attempt to follow it for convenience. You may want to ensure for resumable uploads that you do not automatically follow redirections so as not to lose track of the Location path (and its containing upload key).
Upload file data¶
After securing a file upload key from the server, the client should proceed to upload the file data as the second stage (action 3 in the diagram). The server expects the client to send the file data in an ordered sequence of one or more chunks. Each upload chunk gets formed as POST to the upload path location provided by the server, along with two special HTTP headers:
- Content-Type: {mime-type}
The client must provide the mime-type for the data it wants to upload; the mime-type the client provides here should match the mime-type provided in the first stage.
- Content-Range: bytes {byte-range for chunk}/{total-size}
The client should provide both the byte range carried in the current chunk and the total size of the file (in the format bytes {from}-{to}/{total-size}). (Note that the range expression must be pre-pended by “bytes “, including the space, in lower-case.)
Note that range expresses bytes ordered beginning with 0, where as the total expresses the number of bytes in the file; therefore, the last chunk sent should have a content range header value that looks like {n}-{total - 1}/{total}.
When the server receives the uploaded data, it will check to see if the chunk received is the last chunk (by knowing that it has received the complete range of bytes equal to the file size reported by the client).
If it has not yet received all the file’s data, the server responds with a 308 Resume incomplete response, along with two special HTTP headers:
- Location: {upload path}
This header contains the same location URL as in the first stage, where the last component in the path is the upload key.
- Range: 0-{last byte received}
This header tells the client how much of the file the server has successfully received.
When the server has received the entire file, it responds with a 200 OK to indicate the upload has completed.
Client’s Content-Range header. The client can use the Content-Range header in file upload POST actions in three special ways:
If the client wants to send the entire file in a single chunk, it can omit the Content-Range header. In this case, the server assumes it’s getting all the file data in one POST, and if the number of bytes received doesn’t match the total size declared in the first stage, it will respond with an out of range error (see following).
If the client does not initially know the size of the file to upload, it can send a * character as the {total-size} portion of the header (for example,
Content-Range: bytes 0-50/*
). However, the client must then keep track of the file’s growing size, because it must provide an accurate {total-size} value for the last chunk it sends to the server (so that the server can reliably know when it has received the entire file).If the client wants to check on the status of the upload, it can send a * character as the {byte range for chunk} portion of the header (for example,
Content-Range: bytes */total-size
). If the server has not yet received all the file’s data, it will respond with a 308 Resume Incomplete response indicating the range of bytes received; if the server has received all the file’s data, it will respond with a 200 OK response.
Server expects the upload chunks in sequence. The server always expects the first chunk the client sends to begin with the first byte of the file’s data (range “from” value of 0), and it also expects that each subsequent chunk sent by the client precisely follows in sequence (so, if the first chunk sent is range 0-25, the next chunk’s range must be 26-n).
Because of this, the client must send the file’s data in a serial fashion: it should wait after sending each chunk for confirmation from the server that it has successfully received it. The client should make a sensible decision about the size of the chunks to send based on it’s knowledge of the file’s total size, the network’s reliability, and so forth.
Error handling. Several error conditions can result from the upload protocol:
- File not found
If the client tries to send data to an invalid file upload path, or it has waited too long since the last successful sending of a portion of the file’s data, or it provides an invalidly formatted range expression for the content range header, the server will respond with a 404 Not Found response.
- Length required
If the client doesn’t provide a content length of 0 in the initial POST action to request an upload key, the server responds with a 411 Length Required response.
- Out of range
Several conditions provoke the server to respond with a 416 Requested Range Not Satisfiable response:
The client sends an upload chunk that does not immediately follow the successfully received range reported by the server (for example, the server has received range 0-99, and the client attempts to send range 125-150).
The client sends a {total-size} value that disagrees with either the file size it reported in the first stage of the process, or a previous total size value it sent with an upload chunk.
The client omits the Content-Range header, but has not sent the server all the file data (as reported by the file size in the first stage).
Note that the client can always send a value of * for the {byte range for chunk portion of the content range header to query the server’s status.
- File too large
If the client tries to send more bytes for a file than the maximum size allowed for an upload by the server, it sends a 413 Request Entity Too Large response.
- Completed upload
If the client attempts to send more file data to an upload path for which the server has already sent by a 200 OK response (thinking the upload to be complete), the server responds with another 200 OK. After a certain period of time (the timeout since the last successfully sent chunk expires), the server responds with a 404 Not Found response. After the server understands the upload to be complete, the client may no longer append data to that upload path.
Attach upload to the LMS¶
Once the server indicates it has successfully received a complete file upload,
as the third stage (action 7 in the diagram) the client should use an appropriate
attach file action to instruct the server to attach the uploaded file data to
the LMS. The various product components may provide slightly different routes
for this purpose: for example, ePortfolio provides a
new file attach
action for this purpose, while Learning Environment provides an
attach
action.
As part of this action, the client must provide the upload key to identify the uploaded file data to attach; again, the various product components may expect to receive this upload key value in different ways (LE expects a post form, while ePortfolio expects to receive a JSON structure).