Unable to bulk import a list of urls

dstillman

For a Windows path you need to use two backslashes for each backslash (because backslash is an escape character).

snipespy

@dstillman
That's it! Thank you so much!

bruno30

Hi everyone,

I'm trying to use the javascript code as well but keep getting the same mistake as above:

var path = '/Users/bruno/Desktop/articles.txt';
var urls = Zotero.File.getContents(path).split('\n').map(url => url);
await Zotero.HTTP.processDocuments(
urls,
async function (doc) {
var translate = new Zotero.Translate.Web();
translate.setDocument(doc);
var translators = await translate.getTranslators();
if (translators.length) {
translate.setTranslator(translators[0]);
try {
await translate.translate();
return;
}
catch (e) {}
}
await ZoteroPane.addItemFromDocument(doc);
}
)



[Exception... "Component returned failure code: 0x80520001 (NS_ERROR_FILE_UNRECOGNIZED_PATH) [nsIFile.initWithPath]" nsresult: "0x80520001 (NS_ERROR_FILE_UNRECOGNIZED_PATH)" location: "JS frame :: chrome://zotero/content/xpcom/file.js :: Zotero.File</this.getContents :: line 159" data: no]



What can I do?

bruno30

Sorry, I forgot to say: I used two backslashes as well.

bruno30

I managed to make the code work, but have bumped into another issue: whenever it encounters a broken link it just stops, instead of continuing forward. What should I do?

rlapotre

Hello @dstillman,

I have tried you javascript code for bulk upload from a file of urls.

It's working, but not as well as I had hoped : it's actually adding snapshots from the url's webpages, while what I want is the whole of the references mentioned on each url's webpages (the zotero connector of each of these page is actually offering a list of references to import) to be imported into zotero.
Example of one these pages : https://www.idref.fr/026766701.

Thank you in advance for your help !

fprnt

I tried the initial code that was posted by dstillman and it worked. But if I insert a list of journal article URLs, they will be stored as webpages, not as journal articles. I would need them to be stored as journal articles and, ideally, the abstracts should automatically be downloaded as well. Is it possible to adjust the code in this way somehow?

dstillman

@fprnt: This is all explained in the thread, but as I say, this isn't a supported option and we don't recommend using it, so you're on your own here.

ankanaan

Thanks for posting this. I also have the need for bulk import :-) Helping a friend writing his PhD dissertation. He collected a bunch of references as http links and now needs to add hundreds of such items to his formal reference list via zotero.

As far as I understand and tested the script you posted works by importing everything into the main collection. Is there a way to import the items in the list into a specific collection? He has his references split in several subcollections.

Thanks.

dstillman

@ankanaan: You don't need that. Just sort by Date Added and drag them to the collection.

ankanaan

duuuuh.

thanks very much, that was really easy, problem solved.

Cheers

soumitrakp

@dstillman

Thanks a lot for the javascript.

Could you please update the javascript to:

1. Generate a message for the url whether it was successfully added or not
2. At the end create a new file listing the urls that were not successfully added
3. Add the document in the collection that is selected in Zotero client instead of "My Library"?

soumitrakp

@dstillman: I really need your help at least for 1 & 2 in my previous post. I could create the following version.


var path = '/Users/pals2/ZoteroImportUrls.txt';
var log = '/Users/pals2/ZoteroImportUrls.log';

var status = {};

var urls = Zotero.File.getContents(path).split('\n').map(url => url);

urls.forEach((url, i) => status[url] = "fail");
console.log(status);

await Zotero.HTTP.loadDocuments(
urls,
async function (doc) {
console.log(doc.location.href)
var parser = new DOMParser();
var safeDoc = Zotero.HTTP.wrapDocument(
parser.parseFromString(doc.documentElement.outerHTML, 'text/html'),
doc.location.href
);
var translate = new Zotero.Translate.Web();
translate.setDocument(safeDoc);
var translators = await translate.getTranslators();
if (translators.length) {
translate.setTranslator(translators[0]);
try {
await translate.translate();
return;
}
catch (e) {}
}
try {
await ZoteroPane.addItemFromDocument(safeDoc);
status[safeDoc.location.href] = "success";
return;
}
catch (e) {}
}
)


// Convert the associative array to a string with key-value pairs separated by newlines
const data = Object.entries(status).map(([key, value]) => `${key}: ${value}`).join('\n');

// Write the data to the file
Zotero.File.writeFile(log, data, (err) => {
if (err) {
console.error('Error writing file:', err);
return;
}
console.log('File saved successfully');
});


But it gives error that there is no function Zotero.File.writeFile.

Pls help!

soumitrakp

@dstillman: With little more searching on this forum, I could solve the writing to file issue. However, I get "fail" for all urls. Even through there are urls successfully saved.

Please help with this version.


var path = '/Users/pals2/ZoteroImportUrls.txt';
var log = '/Users/pals2/ZoteroImportUrls.log';

var status = {};

var urls = Zotero.File.getContents(path).split('\n').map(url => url);

urls.forEach((url, i) => status[url] = "fail");
console.log(status);

async function saveStatus() {
// Convert the associative array to a string with key-value pairs separated by newlines
const data = Object.entries(status).map(([key, value]) => `${key}: ${value}`).join('\n');

// Write the data to the file
await Zotero.File.putContentsAsync(log, data);
}

await Promise.all([
Zotero.HTTP.loadDocuments(
urls,
async function (doc) {
console.log(doc.location.href)
var parser = new DOMParser();
var safeDoc = Zotero.HTTP.wrapDocument(
parser.parseFromString(doc.documentElement.outerHTML, 'text/html'),
doc.location.href
);
var translate = new Zotero.Translate.Web();
translate.setDocument(safeDoc);
var translators = await translate.getTranslators();
if (translators.length) {
translate.setTranslator(translators[0]);
try {
await translate.translate();
status[safeDoc.location.href] = "success";
return;
}
catch (e) {}
}
try {
await ZoteroPane.addItemFromDocument(safeDoc);
status[safeDoc.location.href] = "success";
return;
}
catch (e) {

}
status[doc.location.href] = "success";
}
),
saveStatus()
])