1  # Introduction
  2  
  3  The `sym_upload` tool is able to operate in `sym-upload-v2` protocol mode, in
  4  addition to the legacy protocol (which will be referred to as `sym-upload-v1`
  5  for the rest of this document). For now `sym-upload-v2` is HTTP/REST-based but
  6  it could be extended to operate over gRPC instead, in the future.
  7  
  8  # Table of Contents
  9  * [Why](#why)
 10  * [How](#how)
 11     * [Uploading](#uploading)
 12       * [Uploading with `sym_upload`](#uploading-with-sym_upload)
 13       * [Uploading with curl](#uploading-with-curl)
 14       * [Serving the `sym-upload-v2` protocol](#serving-the-sym-upload-v2-protocol)
 15         * [Authenticate using `key`](#authenticate-using-key)
 16         * [Symbol `checkStatus`](#symbol-checkstatus)
 17         * [Upload `create`](#upload-create)
 18         * [Uploading the symbol file](#uploading-the-symbol-file)
 19         * [Upload complete](#upload-complete)
 20  
 21  
 22  # Why
 23  
 24  Using `sym_upload` in `sym-upload-v2` protocol mode has the following features
 25  beyond `sym-upload-v1`:
 26    * Authentication via `key` (arbitrary secret).
 27    * Symbol identifier (product of `debug_file` and `debug_id`, as recorded in
 28  output from `dump_syms`) can be checked against existing symbol information on
 29  server. If it's present, then the upload is skipped entirely.
 30  
 31  # How
 32  
 33  ## Uploading
 34  
 35  ### Uploading with `sym_upload`
 36  
 37  Uploading in `sym-upload-v2` protocol mode is easy. Invoke `sym_upload` like
 38  ```
 39  $ ./sym_upload -p sym-upload-v2 [-k <API-key>] <symbol-file> <API-URL>
 40  ```
 41  
 42  Where `symbol-file` is a symbol file created by `dump_syms`, `API-URL` is the
 43  URL of your `sym-upload-v2` API service (see next section for details), and
 44  `API-key` is a secret known to your uploader and server.
 45  
 46  For more options see `sym_upload --help`.
 47  
 48  ### Uploading with curl
 49  
 50  As an example, if:
 51    * Your API's URL was "https://sym-upload-api".
 52    * Your service has assigned you `key` "myfancysecret123".
 53    * You wanted to upload the symbol file at "path/to/file_name", with
 54  `debug_file` being "file_name" and `debug_id` being
 55  "123123123123123123123123123". Normally you would read these values from
 56  "path/to/file_name", which in turn was generated by `dump_syms`.
 57  
 58  Then you might run:
 59  ```
 60  $ curl https://sym-upload-api/symbols/file_name/123123123123123123123123123:checkStatus?key=myfancysecret123
 61  ```
 62  
 63  And, upon seeing that this `debug_file`/`debug_id` combo is missing from symbol
 64  storage then you could run:
 65  ```
 66  $ curl --request POST https://sym-upload-api/uploads:create?key=myfancysecret123
 67  ```
 68  
 69  Which returns `upload_url` "https://upload-server/42?creds=shhhhh" and
 70  `upload_key` "42". Next you upload the file directly like:
 71  ```
 72  $ curl -T path/to/file_name "https://upload-server/42?creds=shhhhh"
 73  ```
 74  
 75  Once the HTTP PUT is complete, run:
 76  ```
 77  $ curl --header "Content-Type: application/json" \
 78      --request POST \
 79      --data '{symbol_id:{"debugFile":"file_name",'\
 80          '"debugId":"123123123123123123123123123"}}' \
 81      https://sym-upload-api/uploads/42:complete?key=myfancysecret123
 82  ```
 83  
 84  ### Serving the `sym-upload-v2` Protocol
 85  
 86  The protocol is currently defined only in HTTP/REST. There are three necessary
 87  REST operations to implement in your service:
 88  * `/symbols/<debug_file>/<debug_id>:checkStatus?key=<key>`
 89  * `/uploads:create?key=<key>`
 90  * `/uploads/<upload_key>:complete?key=<key>`
 91  
 92  #### Authenticate Using `key`
 93  
 94  The query string arg `key` contains some secret that both the uploader and
 95  server understand. It is up to the service implementer to decide on what
 96  constitutes a valid `key`, how the uploader acquires one, and how to handle
 97  requests made with invalid ones.
 98  
 99  #### Symbol `checkStatus`
100  
101  ```
102  /symbols/<debug_file>/<debug_id>:checkStatus?key=<key>
103  ```
104  
105  This operation expects an empty (or no) JSON payload in the request.
106  
107  This operation should return the status of the symbol file uniquely identified
108  by the given `debug_file` and `debug_id`. JSON schema:
109  ```
110  {
111      "type": object",
112      "properties": {
113          "status": {
114              "type": "string",
115              "enum": ["STATUS_UNSPECIFIED", "MISING", "FOUND"],
116              "required": true
117          }
118      }
119  }
120  ```
121  
122  Where `MISSING` denotes that the symbol file does not exist on the server and
123  `FOUND` denotes that the symbol file exists on the server.
124  
125  #### Upload `create`
126  
127  ```
128  /uploads:create?key=<key>
129  ```
130  
131  This operation expects an empty (or no) JSON payload in the request.
132  
133  This operation should return a URL that uploader can HTTP PUT their symbol file
134  to, along with an "upload key" that can be used to notify the service once the
135  file upload is completed. JSON schema:
136  ```
137  {
138      "type": "object",
139      "properties": {
140          "upload_url": {
141              "type: "string",
142              "required": true
143          },
144          "upload_key": {
145              "type": "string",
146              "required": true
147          }
148      }
149  }
150  ```
151  
152  Since this REST API operation can be authenticated via the `key` query string
153  arg, the service can return a URL that encodes permission delegation to the
154  upload endpoint resource and thereby constrain the ability to upload to those
155  with valid `key`s.
156  
157  #### Uploading the Symbol File
158  
159  Note that the actual symbol upload step is _not_ part of the REST API. The
160  upload URL obtained in the above operation is meant to be used as the endpoint
161  for a normal HTTP PUT request for the contents of the symbol file. Once that
162  HTTP PUT request is completed use the upload `complete` operation.
163  
164  #### Upload `complete`
165  
166  ```
167  /uploads/<upload_key>:complete?key=<key>
168  ```
169  
170  This operation expects a JSON payload in the HTTP request body with the
171  following schema:
172  ```
173  {
174      "type": "object",
175      "properties": {
176          "symbol_id": {
177              "type": "object",
178              "properties": {
179                  "debug_file": {
180                      "type": "string",
181                      "required": true
182                  },
183                  "debug_id": {
184                      "type": "string",
185                      "required": true
186                  }
187              }
188          }
189      }
190  }
191  ```
192  
193  This operation should cause the symbol storage back-end (however implemented)
194  to consume the symbol file identified by `upload_key`. It is up to the service
195  implementation to decide how uploads are assigned `upload_key`s and how to
196  retrieve a completed upload by its `upload_key`. If the symbol file cannot be
197  found, is malformed, or the operation cannot be completed for any other reason
198  then an HTTP error will be returned. JSON schema of non-error responses:
199  ```
200  {
201      "type": "object",
202      "properties": {
203          "result": {
204              "type": string,
205              "enum": ["RESULT_UNSPECIFIED", "OK", "DUPLICATE_DATA"],
206              "required": true
207          }
208      }
209  }
210  ```
211  
212  Where `OK` denotes that the symbol storage was updated with the new symbol file
213  and `DUPLICATE_DATA` denotes that the symbol file data was identical to data
214  already in symbol storage and therefore nothing changed.