[Archivesspace_Users_Group] Trouble With API Endpoint ‘create a new job and post input files’

[Custer, Mark]

Bennett,

Okay, I hope it works, and let me know if not.

Also, sorry for misspelling your name earlier!!!  I’m in the habit of quickly sending emails, forgetting that I can’t edit the message after I hit Send 😊

Mark



From: archivesspace_users_group-bounces at lyralists.lyrasis.org [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org] On Behalf Of Bennett G Poulin
Sent: Thursday, 01 October, 2020 2:05 PM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org>
Subject: Re: [Archivesspace_Users_Group] Trouble With API Endpoint ‘create a new job and post input files’

Thanks Mark!

This looks exactly like what I need. I’ll try it out and let you know.

Thanks again!
Bennett

From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> <archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org>> On Behalf Of Custer, Mark
Sent: Thursday, October 1, 2020 12:01 PM
To: Archivesspace Users Group <archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>>
Subject: Re: [Archivesspace_Users_Group] Trouble With API Endpoint ‘create a new job and post input files’

  [EXTERNAL]
Bennet,

Not spammed out, so no worries!

In case it helps, a couple years ago I was able to get a hokey script set up (with help from others – thanks again, Dallas and Dave!) to download MARCXML files from OCLC and post those to ArchivesSpace using that endpoint.  Here’s a snippet that shows how you need to format the payload (though the filenames don’t matter, I don’t think):  https://github.com/archivesspace/api-training/blob/master/getMARCfromOCLCandPOSTtoASpace.py#L42-L56<https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Farchivesspace%2Fapi-training%2Fblob%2Fmaster%2FgetMARCfromOCLCandPOSTtoASpace.py%23L42-L56&data=02%7C01%7Cmark.custer%40yale.edu%7C2a7c65cfa3014fddbd2508d866347f2d%7Cdd8cbebb21394df8b4114e3e87abeb5c%7C0%7C0%7C637371722957130478&sdata=gPKCFtwNa5MlkJ%2BXJU0a%2BycwfDO3%2FEjliSIpRSJEv7g%3D&reserved=0>  (the important part is adding the files to the “files” parameter as an array of file locations, even if you’re just posting one file, so you’ll need to change how your file variable is assigned, I think)

Mark



From: archivesspace_users_group-bounces at lyralists.lyrasis.org<mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org> [mailto:archivesspace_users_group-bounces at lyralists.lyrasis.org] On Behalf Of Bennett G Poulin
Sent: Thursday, 01 October, 2020 1:46 PM
To: archivesspace_users_group at lyralists.lyrasis.org<mailto:archivesspace_users_group at lyralists.lyrasis.org>
Subject: [Archivesspace_Users_Group] Trouble With API Endpoint ‘create a new job and post input files’

(My apologies if this has been spammed to you. I have been having trouble with the listserv as I am new.)

Hello all,
We are new to the community and started playing around with the API in Python to help familiarize ourselves with it. I can connect and authenticate just fine, even call several of the endpoints without issue, but there is one that I cannot seem to figure out myself. The endpoint I’m trying to connect to is ‘create a new job and post input files’ and is documented here<https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Farchivesspace.github.io%2Farchivesspace%2Fapi%2F%23create-a-new-job-and-post-input-files&data=02%7C01%7Cmark.custer%40yale.edu%7C2a7c65cfa3014fddbd2508d866347f2d%7Cdd8cbebb21394df8b4114e3e87abeb5c%7C0%7C0%7C637371722957130478&sdata=HbOTHEbFuzAn08t6ZtJAEr5m7TnzBYDbUVXiX3QbW64%3D&reserved=0>. I’m trying to use this endpoint to import EAD XML file(s) into a repository. I can use the create jobs<https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Farchivesspace.github.io%2Farchivesspace%2Fapi%2F%23create-a-new-job&data=02%7C01%7Cmark.custer%40yale.edu%7C2a7c65cfa3014fddbd2508d866347f2d%7Cdd8cbebb21394df8b4114e3e87abeb5c%7C0%7C0%7C637371722957140474&sdata=1IZwUozkxF0ocDcUDhtOcGmlOiXg0TBG%2BomlNqpmc2A%3D&reserved=0> endpoint without error, but there are no files, just filenames. I have tried looking through the previous threads here, but it is entirely possible that I missed something.
The Problem:
The problem that I am having is with the encoding for the file(s) that I send along with the post request. I’m bouncing between two errors ‘(RangeError) exceeded available parameter key space’ when I encode with Base64, and ‘(InvalidParameterError) invalid %-encoding’ when I do not. Base64 will bloat the data by 33%, but I’m only trying to send one file at the moment. Is there another encoding that’s preferred, or another way to get the files to the post request?
My Current Execution:
with open(os.path.join(repoDir, "filename.xml"), "r") as f1:
    f1_data = f1.read()
    f1_serialized = base64.b64encode(f1_data.encode())
jobs = json.dumps({
    "jsonmodel_type":"job",
    "status":"queued",
    "job": {
        "jsonmodel_type":"import_job",
        "import_type":"ead_xml"
    },
    "files": { #Omit files for jobs URI
        "filename.xml": f1_serialized.decode('utf-8')
    }
})
createJobs = client.post('/repositories/4/jobs_with_files', jobs)

What I’ve tried:
I’ve tried changing the file open to “rb” and playing with the types of Base64 encoding methods.
I’ve tried adding the files as the third parameter as I’ve seen in other python API requests.
file = {"nmu-lmssking.xml": open(os.path.join(repoDir, "filename.xml"), "rb")}
createJobs = client.post('/repositories/4/jobs_with_files', json=jobs, files=file)

I have tried not encoding the file, but then I get the second error message mentioned above.
I have tried playing with the various file states of read, byte, and encoded data.

Other Relevant Information:
I’m using ArchivesSpace v2.7.1.
AS Documentation Image:
[cid:image001.png at 01D697FD.598162A0]
AS Documentation Text:
curl -H "X-ArchivesSpace-Session: $SESSION" \
  -d 'Example Missing' \
  "http://localhost:8089/repositories/2/jobs_with_files?job={<http://localhost:8089/repositories/2/jobs_with_files?job=%7b>"jsonmodel_type"=>"job", "status"=>"queued", "job"=>{"jsonmodel_type"=>"import_job", "filenames"=>["QJEOU", "PUDYO", "GD44820703", "UK255111B"], "import_type"=>"eac_xml"}}&files=UploadFile"

Endpoint
[:POST] /repositories/:repo_id/jobs_with_files
Description
Create a new job and post input files
Parameters
JSONModel(:job) job –
[RESTHelpers::UploadFile] files –
Integer repo_id – The Repository ID – The Repository must exist
Returns
200 – {:status => “Updated”, :id => (id of updated object)}

AS Ruby JSON Ruby Schema Files for Job and Import Job attached:


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20201001/3a430171/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 20823 bytes
Desc: image001.png
URL: <http://lyralists.lyrasis.org/pipermail/archivesspace_users_group/attachments/20201001/3a430171/attachment.png>