How to batch convert Link to file attachments into copy of file attachment?

Is there a way to batch convert attachments on items that are "Link to file" into "Store Copy of file"?

I migrated from Mendeley, so all my existing items have attachments as "Link to file". I have 500+ items, and I would like Zotero to no longer have a "Link to file", but store a copy that is Zotero internally manages.

I looked into ZotFile, however, I still have not figured out a way to batch convert existing item "Link to file" attachment into copy of file.
  • ZotFile at least _should_ do this when you have it set to "Attach stored copy of file" and then use Manage Attachment --> Rename attachments and I know this used to work. I have a vague recollection that someone was having problems with this some time ago, but definitely worth trying.
  • edited September 24, 2018
    @adamsmith I tested out your suggestion, however, it ZotFile did try to rename/move the files, but it only does it to one item even though I selected multiple items. ZotFile does recognize that I am trying to move/rename multiple items and it even gives me a dialog saying it, but in the end, it will only move/rename one item. In addition, ZotFile actually renamed and moved my "original" PDF files instead of creating copy of it. Not is not good. I created a bug report on GitHub regarding this issue for ZotFile: https://github.com/jlegewie/zotfile/issues/368

    One down side of this method requires to "force" rename the files. What if I don't want to rename the files?

    Just out of curiosity, is it a complex task to implement the conversion of links to copies natively in Zotero?

  • One down side of this method requires to "force" rename the files. What if I don't want to rename the files?
    Zotero also force renames files it attaches, so that's in line with that.

    I don't think this would be crazy _complicated_ to implement in Zotero itself, but since there's currently nothing along those lines at all, it would probably a good amount of work, yes.
  • edited September 24, 2018
    @adamsmith Thanks you for response.

    1) Zotero does not force user to rename the files. Under Edit >> Preferences >> General >> File Handling, I have the ability to disable "Automatically rename attachment files using parent metadata".

    2) Can we have the native support in Zotero for converting existing "Link to file" attachment into "Store Copy of file" attachment be put on a wish list or feature request for Zotero?

    After migrating from Mendeley, I have over 500+ items with attachments as "link to file". All my future additions to Zotero will be copy of files, but the initial items are still stuck with "link to file" thus I can't take advantage of syncing.

    Below are examples of people needing the feature of converting "link to file" into "store copy of file":

    https://forums.zotero.org/discussion/34463/convert-linked-file-attachments-to-stored-attachments
    https://forums.zotero.org/discussion/9043/automatic-conversion-from-linked-files-to-copied-files


    Although ZotFile is a possible temporary work around, it still doesn't properly work due to bugs. Below are some examples of issues:

    https://github.com/jlegewie/zotfile/issues/368
    https://forums.zotero.org/discussion/comment/316079

  • @adamsmith Any chance my previous comment can be considered? Thanks :)
  • Not a part of the code I touch, but the devs that do read every thread, so yes, this is being considered.
  • @adamsmith Thank you for letting me know. You guys are doing a great job! I love Zotero! Thank you for all your efforts.
  • @dstillman @adamsmith In retrospect I realized that feature I am requesting is a very niche case to be considered as a core feature of Zotero. However, it be possible to incorporate this feature in the Mendeley import feature (https://forums.zotero.org/discussion/72260/available-for-beta-testing-mendeley-import)? When a user uses the Mendeley import feature to migrate from Mendeley to Zotero, all the attachment are set as "Link to file" by default. Would it be possible to allow the user to select if they want to migrate the attachments as "Link to file" vs. "Store Copy of file"?
  • When a user uses the Mendeley import feature to migrate from Mendeley to Zotero, all the attachment are set as "Link to file" by default.
    That's actually not the case. In Mendeley files can be stored either in a "Downloaded" folder in the profile folder or elsewhere on the computer in a folder specified in the preferences. When importing from Mendeley, files in Downloaded are copied and stored, whereas files in the designated folder are linked.

    We could add an option to store all files, but it would mean losing the folder hierarchy that Mendeley optionally creates. The alternative for someone who wanted to keep that would generally be to install the ZotFile plugin and set it to use that same folder structure. Of course, linked files in Zotero don't sync, so if syncing is more important than the folder structure — which it sounds like is the case for you — you might choose to store files instead.
  • edited December 27, 2018
    @dstillman Thanks for the clarification.

    Yes, I think giving an option to 'store all files' would be helpful so that users can decide if their priority is syncing or maintaining a folder hierarchy of attachments locally. This would benefit first time migrators on how they plan their future use of Zotero, and they don't have to get stuck with portion of their library being 'linked' and a portion being 'stored'.

    I understand one can use ZotFile to convert linked attachments to stored attachments. However, based on my experience, ZotFile has an issue with batch conversions. Therefore, after a test migration from Mendeley, I wasn't able to convert all my linked attachments as stored copies.

    Previously I preferred to maintain linked attachments so I can have control over my organization and structure. I now prefer to let Zotero store the file so I can use the syncing feature. With zotero-file-hierarchy (thanks to @emilianoheyns ), I can always recovery the attachment folder hierarchy if I ever stop using Zotero.
  • @dstillman Sorry, don't mean to bump this thread. I was just wondering if the feature to have an option to 'store all files' during Mendeley migration will considered for implementation. If your team agrees, do you know if it will be happening in near term? I am just asking so I can plan for migration from Mendeley accordingly.
  • Just a couple day ago I switched on my Linux machine from Mendeley to Zotero. The switch was totally worth it, enjoy working with your software.
    Everything seemed to have worked out nicely. I set up syncing using my Nextcloud account.

    Now on my windows machine I have noticed, that all files werejust linked and not really synced or copied ,because i cannot open any here. I will have to see if I do a completely new import or if one of the suggested workaround works for me.

    I just wanted to add, that it would be helpful to give the option to actually import/copy all files using the mendeley import tool, or at least make the user aware, that files are just linked.
  • I think we'll be able to do something here soon, but we have to figure out a few things first.
  • edited March 2, 2019
    In the latest Zotero beta, the importer will now ask whether you want to copy files to Zotero's storage folder or link to them, and it will copy files by default.

    If you've already imported from Mendeley, this option won't help (unless you clear your database and reimport), but I'm working on a feature that will allow linked files to be batch-converted to stored files. I'll post here when that's available in the beta, which should be within a few days.
  • @dstillman Awesome!!! Thank you very much for this long awaited new feature. I really appreciate your hard work!!!

    I tested out a fresh import from Mendeley with file handling set to "Copy files". The process ran pretty smooth, but took ~15-30min for ~730 items.

    There was one minor issue I came across. After migration, there were few items in My Library which showed the attachment name as "PDF". If I expand an item to show the attachment, it says "PDF", and then when I do right click >> Show File, the actual file name of .pdf is correct and does NOT just say "PDF.pdf". I did a test migration with debugging mode on. I submitted the report, then forced closed Zotero since there is no way to interrupt the migration.


    Report ID: D1210642334
    Screenshot of issue: https://i.imgur.com/prCDzQo.png



  • edited March 3, 2019
    The process ran pretty smooth, but took ~15-30min for ~730 items.
    That seems quite slow — testing here, I can import 90 items in 11 seconds. Does this computer have an SSD or a hard drive?

    We do have further speed improvements planned, though. Among other things, part of the import time is also taking up by full-text indexing, which currently happens during import. The benefit of that is that a full-text search works immediately if someone tries it, but it might be better to perform indexing in the background after the import so that the imported items are available more quickly.
    After migration, there were few items in My Library which showed the attachment name as "PDF".
    That's not actually a bug — the bug is really that some attachments are named after the filename. The attachment title is meant to be separate, and when there's a single PDF, there's not much value in showing the filename, which is usually based on the metadata that's already available in the parent item row. Instead, we just set it to the less-distracting "PDF". But right now that title isn't being properly passed through to all attachments and they're just ending up named after the filename.

    For non-PDF files and cases where there are multiple files attached to an item, using the filename is appropriate.
  • @dstillman
    That seems quite slow — testing here, I can import 90 items in 11 seconds. Does this computer have an SSD or a hard drive?

    We do have further speed improvements planned, though. Among other things, part of the import time is also taking up by full-text indexing, which currently happens during import. The benefit of that is that a full-text search works immediately if someone tries it, but it might be better to perform indexing in the background after the import so that the imported items are available more quickly.

    Yes, I have a SSD, i7 quad core, and 12GB RAM, Win 10 x64. During import, disk usage is below 10% and CPU usage is 30%. I did another retest of import, it took ~8min to for ~304 items to appear in My Library. My 730 item collection results in ~1.5GB data.

    Personally I don't really care about the slow import since it's a one time migration process. I just though I let you guys know about this.

    Once again, thank you very much for this new feature!!!
Sign In or Register to comment.