Best way to allocate storage (sqlite database, storage folder) between SSD and external HD?

Zotero has long been my reference manager and standard advice for my students.

I have over 20 gig (and growing) of PDF files in my storage folder, and 600 meg in the sqlite database. Both in the Zotero folder of my C drive (a SSD), which is getting cramped on my laptop. I always have an external HD with me, where my massive files are stored. I need to free space on C, so I am wondering whether to move the entire Zotero folder over from the C drive to the HD (and then redirect location in Zotero settings), or attempt to retain the database on the C drive SSD for speed advantages, moving only the storage folder.

Any recommendations?

Thanks
  • I would strongly recommend keeping the .sqlite file on SSD and just moving 'storage' if you want to.

    On macOS or Linux you would just do that with a symlink, but I don't know how achievable that is on Windows these days.
  • edited February 23, 2021
    Thanks!

    For an academic, adjusting your Zotero data setup is a bit like brain surgery, so I want to make sure I will be doing this right ...

    So, I close down the app, backup and then move 'storage' folder to new location on external HD, leaving the rest (sqilite DB etc.) as it was on the SSD, and then open the app, and define the new path to the external HD in preferences ("linked attachment base directory")?
  • edited February 23, 2021
    No, absolutely not. This has nothing to do with the Linked Attachment Base Directory. This is about the data directory.

    Zotero doesn't support splitting zotero.sqlite and 'storage' — it just expects those to exist in the data directory, so that's what it needs to look like from Zotero's perspective. As I say, on macOS and Linux you'd use a symlink to do this, but I don't know whether it's possible with an alias or symlink on Windows.
  • Thanks again. I will explore the symlink situation on Windows and do it that way if I feel confident about it.

    Any sense of the speed/reliability penalty of moving the sqlite to the external HD with the 'storage' as well? (I assume it is accessed afresh from SSD/HD each time one accesses Zotero, rather than it being in RAM)
  • If you lots of RAM (e.g., 16 GB or more), the OS file cache might keep the database largely in memory and you might not see as much of a slowdown after the initial startup.

    But it would not be great. Databases in particular strongly benefit from the random-access speed of SSDs.
  • Happy to report that the symlink solution suggested works like a charm. Those who want details, they are below.

    Thanks for the guidance!
    ---------------------

    Am using Windows 7. Rather than doing things by command line I created a hard link using Link Shell Extension (https://schinagl.priv.at/nt/hardlinkshellext/linkshellextension.html#download)
    So, I closed Zotero, backed up and then moved the "storage" folder to a large external HD that is always connected to the laptop, and then created a hard link (called a junction) to that destination folder in the standard Zotero folder. I open Zotero, and it starts fine (somewhat more slowly than usual) but functions as normal and as fast.


    When I examine the properties of that hard link folder (whose icon now denotes that it is a link) I see 20 gig, reflecting the size of the remote target folder. But when I look at the properties of the overall Zotero folder containing it, it is less than 2 gig.

    Thus, I have freed up scarce place on the SSD, and the sqlite database enjoys the speed of SSD. Wonderful.

  • Note also that a forthcoming version of Zotero will have expanded support for keeping synced files “online only”, so you could also preserve SSD space that way.
  • Good to know, thanks. But I assume that anything beyond the free 300 meg would be paid storage, so my 20+gig (and growing) PDF collection would be quite steep....Perhaps I can persuade my university/department to consider the Lab or Institution plans.
  • Storage plans max out at $120 USD annually for unlimited storage.
Sign In or Register to comment.