RIS export failing

tare2

Using Zotero: 5.0.88, Windows.

When exporting a collection as a RIS file with files and notes this fails consistently when exporting it more than once. It works always the first time around but then fails when exporting it again.

My collection has 153 items, some are large (4 Mb) PDF books. All up a 320 MB collection of files.

It says "An error occurred while trying to export the selected file."

But when exporting it again, into a fresh new folder, it fails each time.
The workaround is to make a new collection in Zotero, copy all the items from the old collection into the new collection and then export as RIS again.
So somehow exporting a collection once as RIS makes it impossible to export it again another time.

It gets through about 75% of the items and files on the consecutive export and then fails.

dstillman

Can you provide a Debug ID for an export attempt that fails?

tare2

Here it is: D119902173

dstillman

Can you provide a Debug ID for an attempt that succeeds?

dstillman

@tare2: Also, this debug output is from Zotero startup, and appears to be the first export. Why are you saying this only happens after running the export once successfully?

tare2

Here is another report ID: 1027483991

However, my work around now seems not to be working either. So this is again a shot at RIS export that failed pretty early in the process

tare2

I had been able to do RIS exports successfully before, no problem.
Today, after adding some more items to a collection and adding some notes etc I reported again and the error happened.
I then made a new collection, added the old collection into the new collection and could export as RIS fine.
This worked repeatedly as a workaround.
But now, as you asked me to produce another good is export, it no longer works either. It now seems that I cannot export the collection as RIS at this time.

tare2

Here is another try, again not successful. Luckily I was able to make a successful export earlier today that a co-worker needed.

ID, unsuccessful, just now: 1718441351

tare2

Hmmm... now I ran my workaround again, this time without debug enabled, and it worked...

So to recap: My workaround is to make a new collection, copy the items to the new collection. Then export that new collection. That works, IF Debug is not enabled...

tare2

Now I was able to make a successful export, with Debug enabled:
ID: D834057012

tare2

It seems to have a randomness with it... not a nice bug...

tare2

And another one. D1903213009
This time my RIS export works again and again, with DEBUG enabled and without doing my workaround. I did not need to copy the items to a new folder this time.
Bizarre indeed...

tare2

I will need to leave it for now. I hope you have enough data to work out what happened. If you need more, leave a message. I can get back to it in an hour or so.

dstillman

Yeah, adding items to a new collection shouldn't make any difference — I doubt it was more than just rerunning the export.

Assuming this is a on a regular disk (not some sort of server-mirrored drive), my best guess is that security software on your system is somehow interfering.

tare2

Hmm, na. I have only MSFT security software running that could interfere with security. I have no 3rd party virus checker etc. Its a standalone laptop with a 500 GB SSD, Windows 10. Has been reliable.

tare2

Was there anything consistend that you saw in the bug report?

tare2

Finally I could make it fail again: D1283495363

tare2

I have something, maybe: The length of the name I give to the RIS export!

When I use names such as: "TEST RIS 8" (10 characters) it works.
When I use longer names such as: "STEM and CCE RIS 8" (18 characters) it fails.

Just an idea. I just tried this and consistently it works with short names and fails with long names. So you have a directory file name length overrun, or, due to some file names in the PDFs being long in my collection, there is a path overrun issue at 256 bytes or something that in my case gets triggered with a longer folder name start?

Anyway that's where you need to look for the issue.

I have over 30 years in writing software and started dreaming in Hex writing a massive app for the original MAC qube in 1984... ;-)

tare2

The limit is 10 characters for your RIS export file name: "0123456789" works but "01234567890A" and anything beyond is failing all the time.

So you have a 10 character limit issue that you need to look for in your code. Unless, and that's a possibility, it is connected to a long PDF file. But I know in a second by exporting a collection that has only short file names inside.

dstillman

Windows — not Zotero — has a path limit of 260 characters. In Windows 10 you should be able to remove that limit with a registry hack.

tare2

Ah yes. 260... So what happend is that I asked Zotero to name my PDFs based on the Meta data. That was the last action I did before these issues arrose. This resulted in some long file names made by Zotero for some of the PDFs which prior to that has standard names as given my some of the journals when downloading, which as not useful for making the PDFs self descriptive. I normally just use "authorYYYY'. But when you ask Zotero to make the names from the Meta Data it uses parts of the Title as it seems. And that's where the issue then pops up.
Suggestion: When making PDF names in Zotero from the meta data sort something out that limits the file name to a reasonable length!

tare2

And a registry hack would not be a great idea if I share my RIS with other people as their system would not work....
So better to limit the PDF file name lengths...

tare2

So what do I do now: In Zotero there is no built in UI to rename a PDF manually. Rename from Meta Data is all you can do... Dang.
Can I do a show file in folder, and then manually rename the file there in Windows without breaking the link?
What a pain with some 159 files in that collection to make it more stable to use.
So clearly, in your rename files from Metadata you have an issue with this. Some thinking there needed. You would want unique names if possible if somebody wants to collect all the files in into some other folder outside the Zotero structure perhaps which are not too long.
Also, I normally anyway save all my PDFs in an outside folder and just link to these from Zotero. But for this project I was under time pressure and I allowed Zotero to save the files into its own folder structure with initially the original names from the journal download such as "Full Tex.pdf" which is of course not useful to hand to somebody else.
Thats why I used the "name file from Metadata" feature, which got my extensive collection now into a precarious state...

So if you update your next Zotero please have a look at that issue. It could really nail it for some people badly.

Best!

Thomas

dstillman

Zotero already shortens filename lengths automatically if necessary when renaming or syncing. The problem is that you're exporting to a deep path in your filesystem with a path limit from when filenames were limited to 8.3 characters. We can try to detect this error when exporting and try to shorten the name, but it's 2020, Windows removed this decades-old limitation several years ago, and you should update your system to take advantage of that. Unless you know how other people's computers are configured and where in the filesystem they're going to try to put these files, there's nothing you can do about this — they could put the files 5 levels deeper and run into the same problem if long paths aren't enabled on their computers. 260 characters is just no longer a reasonable limit.

tare2

More and it gets worse: If you look at the Zotero library in Zotero a PDF may be called "Full Text.pdf" but when you then go and do a "show file" it goes to the zotero storage folder and there is a long file name already that is the same that zotero uses when it will do a "rename file from parent metadata". So unless you do this, you have an inconsistent idea about what the actual file names really are.
So in fact, the long file name issue is not arising at the point of doing a ""rename file from parent metadata" but is happening at the point when a file is entered by Zotero into its structure. At that point the actual file name Zotero then chooses is a combo of the author, year, and title and possibly very long. That is hidden from the view because you only see "Full Text" as the PDF file name in the Zotero view in UI in Zotero.
But then later with a RIS export, in particular if it is deeper in my folder structure, bang...

tare2

Ok, yea, perhaps I just need to change that registry entry or restructure my file system. The latter I won't do as it would likely at this time break a heck of a lot of my work badly...

So the registry entry change will the solution for me.

dstillman

If you look at the Zotero library in Zotero a PDF may be called "Full Text.pdf"

No, that's not something that Zotero creates. It might be "Full Text" or "Full Text PDF", but not "Full Text.pdf" if you saved it using any normal Zotero method. The attachment title isn't the same as the filename, and we don't name things to make you think it is.

So in fact, the long file name issue is not arising at the point of doing a ""rename file from parent metadata" but is happening at the point when a file is entered by Zotero into its structure.

By default, Zotero automatically renames the file from its parent metadata when you save from the web, and it sets a descriptive attachment title at that time. The filename would be redundant in the items list, because it's just based on the parent metadata. You can see the filename in the right-hand pane.

tare2

I put that long file enable flag to 1 in the registry and rebooted.
Enable Win32 long paths = 1
But the behaviour did not change in Zotero. The same bug happens when my directory path is too long. So is Zotero not Long Path aware itself?