GitHub API returns invalid file

Question

I am trying to create a file using the GitHub API . The file must contain one byte 0xDD . I use the following command:

 http PUT https://api.github.com/repos/koluch/test/contents/data.txt "message"="New file" "content"="3Q==" Authorization:"token <OAUTH_TOKEN>" 

3Q== is a Base64 representation of the 0xDD byte. This query returns 200 ; however, when I try to get the file using a GET request ...

 http GET https://api.github.com/repos/koluch/test/contents/data.txt 

... it returns the following JSON:

 { "_links": { "git": "https://api.github.com/repos/koluch/test/git/blobs/4e3bc519eef7d5a35fff67687eaee65616832e45", "html": "https://github.com/koluch/test/blob/master/data.txt", "self": "https://api.github.com/repos/koluch/test/contents/data.txt?ref=master" }, "content": "77+9\n", "download_url": "https://raw.githubusercontent.com/koluch/test/master/data.txt", "encoding": "base64", "git_url": "https://api.github.com/repos/koluch/test/git/blobs/4e3bc519eef7d5a35fff67687eaee65616832e45", "html_url": "https://github.com/koluch/test/blob/master/data.txt", "name": "data.txt", "path": "data.txt", "sha": "4e3bc519eef7d5a35fff67687eaee65616832e45", "size": 1, "type": "file", "url": "https://api.github.com/repos/koluch/test/contents/data.txt?ref=master" } 

The content field contains the value 77+9\n , which is not my 0xDD byte. When I use the download url, everything is fine.

Does anyone know what is going on?

Answer 1

I'm still up to date on github support. there is no progress yet. but I think I know the problem. it looks like github is trying to guess the file type of the file and, if it considers it to be a text file, uses the encoding it finds to create a buffer before encoding it as base64.

at least this is what I see from the headers of the response of the raw request. My workaround is to check the file size, and if it doesn't match, reload the blob using the raw API:

  const res = await octokit.repos.getContents(params); let data = Buffer.from(res.data.content, 'base64'); if (data.length !== res.data.size) { log.debug('base64 decoded content length doesn't match: ${data.length} != ${res.data.size}'); log.debug('fetching content from ${res.data.download_url}'); const result = await rp.get({ url: res.data.download_url, resolveWithFullResponse: true, encoding: null, }); data = result.body; }