Problems with linking files + merging duplicates

emwmorgan

I accidentally deleted all of my PDFs in a folder I was copying from my desktop to Zotero; I must have clicked and held the key commands in the wrong order or let go slightly too soon. Luckily I had a backup from this morning and was able to restore the files locally, but now they're mixed up with the renamed files from Zotero (I tried reversing it first by dragging and made a mess instead).

In Zotero, I now have between four and six different copies of the same citations with copied PDFs and linked PDFs and I'm manually merging them all--but that requires twice confirming that I want to merge, whereas the feature that got me into this mess didn't so much as generate a single prompt for what I wanted to have happen! Seems like a pretty big design flaw that could use some fixing.

Is there a better way to do this? I have about a dozen folders left to go, with hundreds of PDFs nested inside.

dstillman

whereas the feature that got me into this mess didn't so much as generate a single prompt for what I wanted to have happen! Seems like a pretty big design flaw that could use some fixing.

Not sure what you mean here, or what you're even trying to accomplish. What "feature" are you referring to?

Note that we recommend using stored files. Linked files are an advanced, much more error-prone configuration.

Merging duplicate items also simply requires clicking one them in the Duplicate Items view and clicking the merge button.

emwmorgan

I would love to use stored files, but I can't afford to pay for the storage capacity. The feature I referred to is the ability to import large numbers of linked PDFs and retrieve their metadata, which is outlined in the Retrieve PDF Metadata page specifically aimed at new users. This involves holding down two keys while dragging and dropping the files over with the mouse, and there is a note explaining the unreliability of this approach, particularly for Mac users. The note at the end emphasizes that this is not the best way to import files in general, but "this feature can greatly facilitate importing large existing libraries of PDFs". And when it works, it really really works.

I'm not a new user (going on five years over here, and I positively evangelize Zotero) but I haven't yet adopted a "save to Zotero first" workflow. This was one step towards adopting that workflow. I was in the process of migrating all my PDFs into Zotero as stored files when I ran out of space, and so I removed the stored files and began trying to create citations for each of my PDFs. Even when I have non-OCRed PDFs, it is still much more efficient than creating each citation individually--except when importing this set of files went dreadfully wrong.

I understand that stored files or copies are the default. It would make things much easier if I could change that default, because an impossible to gauge margin of error is what led to my files becoming stored files instead of linked files and created a real mess of my Zotero library as well as my desktop storage system.

I know how to merge duplicate items, and it is a good thing that I'm asked to confirm that I want to merge the items. That was my point: a similar prompt to confirm the format (probably not the right term, but I mean linked or stored files) of the imported files would have helped to forestall the problem I had. I had to merge items over 200 times, and I still have to go through each of those merged items manually to delete each of the stored files within the merged parent items.

I also had to restore my files using a backup, as there was no obvious way to reverse what I'd inadvertently done, because Zotero had automatically renamed my meticulously named PDFs and I could no longer easily determine what was a duplicate. I've since changed the file handling in the General pane so that attachments aren't automatically renamed using parent metadata, but because I hadn't tried to link files before I hadn't noticed that this default existed. Had I not already developed the habit of observing very consistent naming conventions that might have been very useful, but as it was, it completely disrupted my existing system.

I'm wishing now I'd been clearer in my initial comment, but that was mid-accidental mass deletion and I was more than a little panicked. My question still stands, though: is there a better way than drag-and-drop (with the associated risk of accidentally importing all the files as stored files rather than linked files) to import hundreds of PDFs as linked files in order to retrieve their metadata and create corresponding citations? I'm already moving folder-by-folder, so I have a built-in way to check if the import has gone awry and I'm never trying to get Zotero to import more than about 75-100 PDFs at a given time. If that number should be much smaller, that would be great to know.

dstillman

You're conflating things, though — adding large numbers of items at once and running metadata on them is totally separate from using linked files, and using linked files is totally separate from the specific method of holding down a modifier key to add them. The Retrieve PDF Metadata page you quote doesn't say anything about modifier keys, and in fact explains that you can use "Link to File" from the New Item menu to create linked files. It sounds like you're referring to the Attaching Files page, which just provides general guidance on adding linked files, and drag-and-drop is just one of the three documented ways to do it.

Also note that most people who use a linked-file workflow use the ZotFile extension, as mentioned in Stored Files and Linked Files, rather than linking existing files manually. You can also configure ZotFile to automatically use a naming scheme of your choosing rather than naming files manually — we strongly encourage people not to waste time naming files manually (and a future version of Zotero will include more flexible renaming options).

There's also a plugin that can mass-import files from an existing folder structure, though we can't help with that. (A similar feature will be included in a future version of Zotero.)

emwmorgan

I don't believe I am conflating things: the metadata runs automatically upon dragging + dropping the items; dragging + dropping is the method for adding multiple linked files, which must be done while holding modifier keys. I pointed out the large numbers because I know that is a possible factor--the opposite of conflation. I did refer to multiple pages in order to figure out how to do what I needed to do. So yes, I looked at all three of those pages and worked out what seemed like a reasonable series of steps from all three.

Thank you for pointing out the plugin and Zotfile extension. I haven't looked into the Zotfile extension yet; it didn't seem to me to be particularly necessary for what I wanted to do. The future features sound very useful, but don't exactly help with my current situation.

My objection (not being able to change the default from stored files to linked files), suggestion (there should be a prompt to confirm the format as linked or stored files or a way to change the settings), and question (what should I actually be doing to accomplish this task of importing ~400 PDFs or even just their citations if not this) still stand.

Thank you for your suggestions. I really do appreciate them.

dstillman

I don't believe I am conflating things

But I mean you just are — you claimed the Retrieve PDF Metadata page is about adding linked files via drag and drop with the modifier key, when 1) it's not specifically about linked files, 2) it doesn't mention the modifier key at all, and 3) it specifically explains another way to add linked files. Again, Retrieve PDF Metadata doesn't have anything to do with choosing stored files vs. linked files, and using drag-and-drop with a modifier key is just one of various ways of adding linked files. I'm not sure why you're arguing about this when I'm taking the time to clarify the differences here and point you to the relevant documentation.

there should be a prompt to confirm the format as linked or stored files or a way to change the settings

As explained on Stored Files and Linked Files, we don't recommend linked files for most people, and most people who use linked files use ZotFile, so we're certainly not going to add a prompt on every drag of a file to Zotero, and we're not inclined to make them settable as the default without a plugin that automates the process.

what should I actually be doing to accomplish this task of importing ~400 PDFs or even just their citations if not this

Except I explained three different ways to do this? 1) Link to File, 2) ZotFile, or 3) the Folder Import plugin. I'm not sure why you're saying I didn't answer this.

emwmorgan

I think we're talking at cross purposes here. I think you've misunderstood what I'm saying and I've misunderstood you. I've written the below to clarify, not to dispute anything.

You're right that I conflated the content of the pages; it was not clear from your response which aspect of the entire story you meant to refer to when you wrote "things". I would have linked to each page, but am unable to do so (at the very least it is not obvious from my end) and frankly it seemed excessive. I further understand why you might not want to add a prompt every time someone drags a file into Zotero.

I am not saying you didn't answer the question. I thanked you for the suggestions! And I will thank you again for numbering them in your response; that was clarifying, because I hadn't registered the reference to Zotfile as a suggestion for the specific problem of managing the citations of existing PDFs. The Zotfile suggestion seemed to be about renaming files, not importing citations for my existing PDFs.

I was trying to link to the files in the first place. Apparently I did that wrong, even though I was following the instructions in the Stored and Linked Files page as precisely as I could. So when I wrote that my question still stands, it's because I wasn't sure what I should be doing other than what I had already tried to do.

The plugin seemed (and still does seem) like a possibility; thank you for bringing it to my attention. After the mess I made just trying to work within Zotero, I am not eager to complicate matters with an unfamiliar-to-me plugin. I look forward to the future version of Zotero which incorporates that function.

I obviously don't know as much about this as you do, so there are almost certainly things that seem self-evident to you that are not self-evident to me.

Thanks for your help.

adamsmith

I was trying to link to the files in the first place. Apparently I did that wrong, even though I was following the instructions in the Stored and Linked Files page as precisely as I could.

To be clear, there are two ways of doing that, though, and using drag&drop + keyboard modifiers (which I understand you to have used) is the more error-prone option: If you instead use "Link to File" option under the "New Item" (green +) menu, you will know for certain that you're linking to files (and you can select multiple files in the selection dialog)