Handling of PDF files

mkostura

Just wanted to provide an observation regards the download of PDF files from journal sites. I seem to have an unpredictable problem with downloading a journal article along with the PDF file. The PDF download will fail from the journal page. I can open the PDF file and the page will be recognized by the Zotero connector (all good) but it will not download the file into the Zotero database. I have attempted the download with both Firefox (default) and Chrome. This particular link provides a good example of the issue that I am having. http://www.pnas.org/content/113/19/5263

adamsmith

Works for me. Which version of Zotero and the connector?

How are you authenticating for access to the PDF (i.e., are you on a university campus, do you go through some kind of proxy, do you log in to the website)?

Also, when you say "unpredictable", does that mean that you can sometimes download the file and sometimes not? Do PDF files download for journal articles ever?

mkostura

I have version 5.0.35 for the connector and 5.0.37 for Zotero desktop

I am not on a university campus and am not using a proxy

Right now it is plenty predictable....does not work.

mkostura

I checked the error log Report ID: 1436573065

Seems that there is some kind of javascript error coming up.

mkostura

And here is a Debug Log ID # D303469376. Again this shows a failure in Java.

dstillman

Can you provide a Debug ID from Zotero (rather than the Connector)?

mkostura

The Debug ID is D835778677.

mkostura

And a report ID 1701697354

adomasven

Error:
Win error 5 during operation move on file C:\Users\[UserName]\Documents\zotero\storage\tmp-msQ2WM (Access is denied.)

Zotero does not have sufficient permissions in C:\Users\[UserName]\Documents\zotero\storage. That could be due to initially running Zotero as administrator or some general permission messiness within Windows. You have to ensure that the user account that you run Zotero with has read and write permissions in that directory.

mkostura

There is only one user on this machine and that user account certainly has admin permissions. Windows (or Zotero) also inserted some unknown user ID's. I have allowed full read-write permissions across the board for all listed users and still no go. I do not recall installing Zotero as an administrator.

adamsmith

might also be security software -- hard to say more, you see the error above: it's an access denied message from the operating system.

What sort of set-up is this? Just a regular personal computer?

Windows (or Zotero) also inserted some unknown user ID's.

user IDs as in there are Windows accounts on the computer you don't recognize? Certainly not Zotero's doing, but that would be a cause for concern more generally.

mkostura

Certainly see the error. But this failure only occurs when using the Zotero Connector and only when trying to download PDF files. I can drag and drop PDF files into Zotero with no problem. Also I can save PubMed pages no problem. This computer is a regular personal computer using Win10 Home.

adamsmith

I'd guess security software then (PubMed doesn't actually attach files, so that's different)

mkostura

I had Microsoft work with me to debug potential security and permission issues. From their perspective, the folder permissions are configured properly. File operations in Zotero seem to be working. The program can write to the directory (drop a PDF file into Zotero and it will correctly place the file into a new folder in the storage location) Zutilo can rename that attachment and place it into another directory. So, it seemed to Microsoft, the permission on the folder are working as intended. But none of this can occur through the browser. The question is why the browser connector is unable to access the storage location and instead reports a permission failure. The connector issue occurs in both Firefox and Chrome so it is not dependent on the browser. MS was at a loss to explain the behavior and nothing in the event logging provided any clues.

There was one question that MS had regards the behavior of the software. What persistent links are created to the destination storage folder? We attempted to uninstall and delete all remnants of Zotero. To save the current image we tried to rename the directory (called it zotero_old) but that failed. Seems that there is another process with a link to the folder. Even with the browsers and Zotero turned off, the link persisted. The only way to break the link was to restart the computer or uninstall Zotero. That was somewhat peculiar to them.

So far I have:
Reinstalled Zotero desktop and connector - twice.
Refreshed Firefox and reinsalled all add-ons
Removed security software
Performed an in place repair of MS Windows 10 - no change for any of these items.

mvkarra

I seem to be having the exact same problem (just included a new post on the forum).

mkostura

Thanks. Checked it out but not sure if it is the same issue. I have pretty much exhausted all Windows permission fixes. I even activated the elevated administrator user and installed zotero and zotero connector for that user. Using the elevated administrator user should eliminate all permission issues. But that was a no go. Microsoft is essentially saying that this seems to be a user software issue and unless I want to go through the trouble of wiping my computer and then reinstalling all software then I have to live with the problem. Oy.

dstillman

You have your Zotero data directory in a custom location. Are you by any chance syncing Documents via OneDrive or another cloud storage system?

Have you tried moving your data directory to the default location of C:\Users\:username:\Zotero?

mkostura

My local directory is not on a cloud device of any kind.

I have uninstalled and reinstalled the Zotero software a number of times. I have allowed it to use the default location and in those instances the sync process with the Zotero servers has worked flawlessly. So it seems the Zotero software is able to write to the local storage directory. But the Zotero browser connector has not worked under any circumstance. Still the same error: access denied.

dstillman

It's pretty clear that this isn't a standard permissions issue — the error is occurring after Zotero has already written the files to a temporary directory in 'storage' and is trying to move them to the final location. So it's not a question of Zotero having write access to that folder. (The browser connector doesn't touch the filesystem, by the way. Attachment saving happens through Zotero itself.)

Most likely something is opening the new files after they're created in the temporary location, and on Windows it's common for files to be opened in a way that blocks writes. (E.g., if you have a PDF open, you often can't write to it or move it.)

Usually this is due to security software, but it could be something else. Have you tried with your third-party Zotero plugins disabled?

mkostura

I have tried to run the software with all plugins disabled. No change. I have also gone further to diagnose what is going on. I have run a process monitor during the Zotero connector initiated download. You are spot on correct that the connector is opening the storage folder and is writing a temporary folder to the storage location. That is successfully accomplished. So there does not appear to be a typical permission issue. But the next event is not described as a move but a rename of that folder. Specifically the process is

High Resolution Date & Time: 3/23/2018 12:36:08.3121537 PM
Event Class: File System
Operation: SetRenameInformationFile
Result: ACCESS DENIED
Path: C:\Users\Matthew Kostura\Documents\Zotero\storage\tmp-jn0ex7
TID: 21380
Duration: 0.0003793
ReplaceIfExists: True
FileName: C:\Users\Matthew Kostura\Documents\Zotero\storage\XVTXVAS8

There is no security program process or thread associated with any of the operations that I can see. I believe the error I am seeing is a failed interaction of the Windows Explorer file system and Zotero.

dstillman

Yeah, rename and move are the same thing in technical terms. There's probably a tool that can show what program, if any (and perhaps even Zotero), is holding the file open while the rename happens. (The Linux/macOS equivalent would be lsof, which one could run in rapid succession to try to debug it.)

I'm also curious if the delete that happens on failure is happening successfully. (It should be, based on the lack of another error in the debug output.)

mkostura

The temp file/folder is not present.

The process monitor I am using provides fairly extensive tracking and so far I have not noted another process especially a security program that is accessing the file.. But I could be missing something.

JustASKw2w

This isn't a super technical contribution. But I had a couple different pdf viewers installed in Chrome, and if I opened a pdf with one particular extension, even if Zotero was online, nothing would happen when I clicked "Save to Zotero". I removed those extensions and now am able to save pdfs to Zotero from Chrome without issue.

The Adobe Acrobat extension seemed to play nice.

mkostura

Problem isolated. After a fairly involved diagnosis, it did indeed come down to a piece of security software, but not my primary firewall, antivirus etc. I also make use of a program called MalwareBytes, a piece of software for preventing web based attacks. Zotero was set as an exclusion for the software but for some reason the software still blocks the connector from working.

dstillman

Yep — someone else identified MalwareBytes as a cause yesterday. Thanks for letting us know.

mkostura

Thanks and you are welcome. I found the link by running the machine in safe mode. When I started to look at my security software that is what came up. There is no solution in the settings in MalwareBytes. I even excluded the offending storage folder from scanning and that still did not help. The real time protection must be turned off or the program cannot be loaded....or as I have done uninstalled from the system.