Another vote for relative paths. I run Windows XP in the office and Windows 7 at home, and 7's "users" versus xp's "documents and settings" file system makes it impossible to even create identical paths across operating systems.
I downloaded Zatero this morning at work and ran into the "relative-path-to-linked-attachments-problem" right away after synchronizing my external hard drive with my home PC to continue my literature management.
I got the same folder structure at work (WinXP), on the flash disk and at home (Win7). I set my "Zotero-Data-Folder" to a destination within that structure, so that with relative links to the attachments, all documents should be reachable. And I would be able to synchronize my Zotero-library with the same sync-routine that I am used to...
...So add one more vote for the Feature Request ;o)
By the way, I couldn't find a ticket for that yet in the Zotero Issue Tracker (https://www.zotero.org/trac/). As guideline 2 restricts new ticket creation to core developers and admins, I wanted to ask if this "feature" is actually planed or if I have overseen an according ticket.
the core development team has very limited resources. Zotero is open source and anyone is welcome to submit a patch to zotero-dev or to trac that will add this functionality.
(for what it's worth, Dan has also pointed out in the past that they don't make decisions based on which thread has the most +1s - other concerns, such as grant requirements and broader design concerns take precedence.)
If you have a forum with a category "feature requests", you're not going to be able to keep users from requesting features and indicating what they would need most.
I'm not trying to keep anyone from doing anything. But I think it's important for people to realize that these type of 'me too' requests have almost no impact on the actual development process. People seem to believe that they do - and so they get upset when "frequently requested" features don't get implemented as quickly as they would like - so I think it's useful to clarify.
Remember that the resources for the core development team come from grants and zotero file storage subscribers. Features that are not requested by either of those are going to slip further back in the development cue (unless a third party picks them up such as csl 1.0/citeproc).
I'm not arguing against your point. I happen to know that it is like you say. And some users following this thread might now know it too. But I'm just saying that having a Feature Request category seems like asking for problems if in actual fact there are no possibilities to do something with these requests.
Some requests do get implemented, some small ones very quickly. In other cases Dan has mentioned how discussion about the form a potential feature can take are very useful for developers when they implement it - that seems to be the main interest from the developer side. From a support side about 70-80% of new posts in Zotero are answered by pointing out how something is already possible or how something is unlikely to ever be implemented - I think that's useful, too.
So I do think this part of the forum plays an important role. I agree to a good extent with you that it would sometimes be nice to know where a feature is "in the queue", but what I'm picking up from dev is that with such a small team, software development is just too unpredictable to make reasonably reliable statements about that, so rather than promise something and be 2 years off, devs have decided not to comment unless something actually moves.
I understand that changes being made or not being made is a function of available resources (which are limited), and not of how many "me too" messages are posted here. Still, can somebody more knowledgeable about Zotero development explain to the rest of us what makes adding the relative path storage option such a big challenge? I am not a developer, and my knowledge of programming is very limited, so I am quite willing to accept that there may be reasons for this feature not becoming available, even after being requested for the last 2.5 years. But it would be nice to know, for our education if nothing else, what those reasons are. Thank you.
It is very disappointing that nobody from the Zotero team bothered to respond to questions/requests here with any kind of explanation, other than "the developers' resources are limited". They sure are. But it would be nice to know why this feature, requested by many users through the last 3 years, is of less importance than other issues that do get addressed in newer versions of Zotero.
In the meantime, here is a workaround I came up with, for myself. Probably not the most elegant solution, but in case some people may find it useful:
For those materials that I want to store outside of Zotero, I use the URL field of a corresponding Zotero record, to store the directory path to the folder on my hard drive where those materials are. For example, all my electronic books on Statistics are in the folder C:\MY-EBOOKS\_Statistics, and it is C:\MY-EBOOKS\_Statistics that goes into the URL field for each of these books' records that I have in Zotero. Then, when I want to go from a Zotero record to the actual e-book file, I click on the URL in Zotero, and Firefox presents all the files in that C:\MY-EBOOKS\_Statistics folder as a webpage. If you have a certain naming scheme that you are following, for your files (e.g. Author-Title-Year-ISBN, or something like that), then it is easy to scroll down the list on the generated webpage, get to the file you wanted to open, and click on it. Basically, it takes two clicks to open the needed file: a click on the URL field in your Zotero record, and then another click on the webpage generated by Firefox.
One disadvantage here is that Zotero records have only one URL field (which is quite unfortunate, since it creates a problem under other scenarios as well). If you use up the URL field for storing your hard drive folder path, but still need some other web address added to the record, you don't have a "clickable" field to put it in. Still, you can add that other URL to the record by adding it as an attachment: right-click on the record line in the central panel, select Add Attachment, then Add Link to Current Page. Alternatively, if I know I won't need to go to that web address often, so I don't need it to be "clickable" that badly, I put the URL(s) in the Extra field of the record.
I can't say I am absolutely happy with this solution -- there is always the risk that some changes in the future versions of Zotero may make this workaround obsolete, or more difficult to use. But that's the risk you always face when coming up with your own shortcuts / macros / other ways of working *around* software features. Still, I am sticking to it for now, since I've invested too much time and effort into building my Zotero collection, and switching to Mendeley / Wizfolio / anything else would be more resource-intensive.
If you find this solution of use but have ideas on how to improve it further, I'd appreciate hearing from you.
Sorry about the long post above. Still, one more thing I should mention, to address the issue of "relative paths (and no drive letter)" brought up above (by Spicedreams, Nota Bene and others). I tried to work around the issue of different Windows OS (XP, Vista, 7) by moving these storage folders to the root directory level (to avoid the confusion of User, Default User, and all other variations, and also to make the whole directory path shorter). The sync software/service that I use on all of my computers (SpiderOak) is set to backup & sync the same C:\MY-EBOOKS directory that I have created on all of my computers. So it doesn't matter which Windows OS each of them has installed.
There are a number of reasons why some people may not like moving their important content files out of their user directory, but it works for me... FWIW
I tried brows the database zotero.sqlite in SQLite Manager and find the table storing those pathes. But I don't know how to change them. Some IT guru maybe develope a small tool to make it.
Moving things to the root directory is not an option for people in any sort of "corporate" environment, including at a university with a standardised PC setup.
I am keeping my pdfs on a dropbox online folder, so I can provide the zotero item with an absolute url. Has anyone tried this?
Of course there are issues: mainly, that zotero only supports one url per document, and that copyrighted materials (such as proceeding files provided in conferences) are *theoretically* accessible to everyone (everyone knows the url..) this way!
I absolutely vote for relative path support.
Another vote for relative path and linking rather than storing as a default behaviour. IMO Zotero needs to get past the monolithic storage philosophy to grow. Nobody wants to be locked into a proprietary storage model as the risk when it comes time to change software down the track is just too great.
it's not proprietary. Nothing in Zotero is.
Also, the crucial feature avoiding lock-in is the availability of good export options - Zotero offers all standard export formats as well as an open standard, which allows e.g. one click syncing to Mendeley.
Compared to data export, the file structure is just not a very important issue.
Zotero offers all standard export formats as well as an open standard, which allows e.g. one click syncing to Mendeley.
To be fair, reading directly from Zotero's internal database, while easy because Zotero's code is available for perusal and the database is a standard Sqlite database, shouldn't really count as an "open standard". Reading from the internal item representations, while easier with Zotero than say Endnote, is still not a standard. It's not documented anywhere outside of the Sqlite schema, etc.
But I still agree 100% with adamsmith on the core issue-- Zotero tries to help data keep moving, keep being exchanged. The internal database structure and the file structure Zotero uses should not matter to you. Zotero will gladly export the files and data in a number of standard formats when you decide to do so. I suppose the important thing to remember is that Zotero is not designed as a file organizer. It is designed to be a flexible cross-platform research environment, so development focuses on ways to do that well.
Home-dir relative storage of linked files (the topic of this thread) would be a good improvement-- but the lack of this feature says _nothing_ about Zotero's philosophy.
Just to clarify - when I said "Nobody wants to be locked into a proprietary storage model " I mean a model where the storage is not intuitive or useful once the framework that built it is taken away - proprietary isnt the right word.
In effect the use of directory names that are jumbled character strings means even though the files are intact, they are not stored in any logical way if you tried to navigate the tree from outside of the software. This is a negative in my book as it means if/when I want to stop using zotero I have an impossibly large re-organising task to re-acess those PDFs that in all honesty could never be accomplished. This is why my preference would be to store the files elsewhere in some other logical (to me) format (alphabetical or organsied by subject) and then link the whole lot into zotero. For references like web pages I can see where saving a snapshot into zotero on the fly is desireable. Nearly all my library is PDFs of scientific articles stored already in a format where I can access them independently to any software and linking them into zotero is the logical choice but with changes in storage, (e.g. local pc to fileserver, or adding a new drive) the option to move the storage tree's path without breaking the links is highly desirable.
I aggree having relative paths. This query has to be seen in the light that tablet PCs like the iPad become increasingly popular. Synchronization and online storage services such as Dropbox become increasingly important to get the document seamlessly onto the tablets and to synchronize files between different (PC) computers. Paying for Dropbox I am not interested in paying for Zotero online storage again. Dropbox I can use for much more than just for papers, it is cheaper, and synchronizes to tablets. Different computers have different storage locations - which can be solved through relative paths. I cannot be that difficult to implement.
In an earlier post you wrote that you are dependent on the money from sponsors and that they determine priorities of further developments. What's about donations? I think people would be quite willed to donate some money for a specific feature. Donating without knowing that the things most desired are implemented is not very encouraging.
I agree with "grubs Mar 3 2011" re: relative paths. I maintain my documents (jpgs, pdfs, etc.) in a separate file structure that can be moved, saved, shared, etc. easily. I've changed citation management and genealogy programs over the last 15 years while maintaining actual documents as is. This is an important feature for those who have to move to another program or OS updates/change. Endnote doesn't do it. That why I'm moving to Zotero and hoping that it will provide robust support for attach link to file in the future. My current genealogy program The Master Genealogist does allow a complete update of all linked documents. This was a simple way to move from WinXP to Win7 and go with the new user documents file structure. Endnote required relinking each document. That's when I began looking at Zotero. Beyond local PC use, saving the documents in the cloud or via a group server is essential for many researchers.
One other comment. I really like the new Zotero 3.0b2.1 standalone and have decided to go ahead and make the switch from Endnote. In addition to the attached link storage model, it appears that this beta version allows only in-Zotero storage for drag-drop of pdfs/jpgs. Is there currently or a planned feature that drag-drop could default to attach by link by file vs attach/store copy of file (in Zotero)? Perhaps as a preference option?
my understanding about links by drag&drop is that it's planned in general (either as an option or by keyboard shortcut - i.e drag+shift creates a link or so) but like relative file storage it wouldn't appear to me to be currently high priority.
I assume patches would be gladly accepted though.
Maybe of interest to others posting here: many people's needs may be served by Zotero plugins such as Zotfile and Zotfile reade, esp. wrt. syncing to tablets. Zotfile now also supports some version of relative paths.
I'm really interested into relative and I think I have some kind of solution.
I haven't tested everything yet and it may not work with platforms other than windows but it may give ideas on ways to work around the problem. My solution is to make your OS make the "relative path" by mapping a network drive.
Say your pdf library is in C:\documents and settings\user\my documents\references. Map the network drive (for exemple drive Y:) to be C:\documents and settings\user\my documents\references.
Then if you have xx.pdf in your library and you attach it to an element of your zotero library as Y:\xx.pdf and you make the same directory and network map drive on the other computer, the path always stay Y:\xx.pdf. Then if you make the mapped directory sync with dropbox or other web service and you sync your database through zotero syncing service, everything should work ok.
I just made up this solution this morning and haven't tested yet on a second computer. But with "show file" in zotero, it really point to Y:\xx.pdf and not the the whole path mapped by Y:.
I have some strategies to overcome this problem. Maybe it helps...
(i) link files in your personal way and change the sqlite database to point to that files: %path to zotero profile%\zotero.sqlite - table "itemAttachment" - variable "path" - it has the path to files, that files in zotero storage are named "storage:filename.pdf", files in your OS are named "C:\way\to\file\file.pdf" - sync will not work to linked files, you have to sync on your own (dropbox?), but in each machine you have to put the files in same way. - different OSes problem. - cons: hardworking -> open sqlite db and change each folder.
(ii) you can change the dynamic named folders inside zotero storage. - in zotero.sqlite, table "item" every attachment has a "key", this key is the path in storage directory, if you change the key and the folder to the same way, you may put more than one file per folder. - sync will work because the files are inside storage folders. - pros: you stay with your own selflogicnamed folders - cons: hardworking -> open sqlite db and change each key.
(iii) the easy way: Install wampp, xampp, ....., "portable" - run xampp portable - create a symlink in htdocs inside xamp to point to your folder structure. Ex.: C:\xampp\htdocs\Reference <=> J:\Files\any\PDFs. (http://www.howtogeek.com/howto/windows-vista/using-symlinks-in-windows-vista/) - In Zotero, create a reference and "Attach link to url". Ex. http://localhost/Reference/filename.pdf - Thats it. - Pros: This solution can be used in any OS (ok, I didn't test any xampp solutions in linux either in mac, but I think it's possible). Your folder structure doesn't change, only symlinks changes... - cons: lost pdf index.
Upgrade Storage
I downloaded Zatero this morning at work and ran into the "relative-path-to-linked-attachments-problem" right away after synchronizing my external hard drive with my home PC to continue my literature management.
I got the same folder structure at work (WinXP), on the flash disk and at home (Win7). I set my "Zotero-Data-Folder" to a destination within that structure, so that with relative links to the attachments, all documents should be reachable. And I would be able to synchronize my Zotero-library with the same sync-routine that I am used to...
...So add one more vote for the Feature Request ;o)
By the way, I couldn't find a ticket for that yet in the Zotero Issue Tracker (https://www.zotero.org/trac/). As guideline 2 restricts new ticket creation to core developers and admins, I wanted to ask if this "feature" is actually planed or if I have overseen an according ticket.
Cheers, Tobi.
Is it really that hard to add?
Zotero is open source and anyone is welcome to submit a patch to zotero-dev or to trac that will add this functionality.
(for what it's worth, Dan has also pointed out in the past that they don't make decisions based on which thread has the most +1s - other concerns, such as grant requirements and broader design concerns take precedence.)
Remember that the resources for the core development team come from grants and zotero file storage subscribers. Features that are not requested by either of those are going to slip further back in the development cue (unless a third party picks them up such as csl 1.0/citeproc).
In other cases Dan has mentioned how discussion about the form a potential feature can take are very useful for developers when they implement it - that seems to be the main interest from the developer side.
From a support side about 70-80% of new posts in Zotero are answered by pointing out how something is already possible or how something is unlikely to ever be implemented - I think that's useful, too.
So I do think this part of the forum plays an important role. I agree to a good extent with you that it would sometimes be nice to know where a feature is "in the queue", but what I'm picking up from dev is that with such a small team, software development is just too unpredictable to make reasonably reliable statements about that, so rather than promise something and be 2 years off, devs have decided not to comment unless something actually moves.
I understand that changes being made or not being made is a function of available resources (which are limited), and not of how many "me too" messages are posted here. Still, can somebody more knowledgeable about Zotero development explain to the rest of us what makes adding the relative path storage option such a big challenge? I am not a developer, and my knowledge of programming is very limited, so I am quite willing to accept that there may be reasons for this feature not becoming available, even after being requested for the last 2.5 years. But it would be nice to know, for our education if nothing else, what those reasons are. Thank you.
In the meantime, here is a workaround I came up with, for myself. Probably not the most elegant solution, but in case some people may find it useful:
For those materials that I want to store outside of Zotero, I use the URL field of a corresponding Zotero record, to store the directory path to the folder on my hard drive where those materials are. For example, all my electronic books on Statistics are in the folder C:\MY-EBOOKS\_Statistics, and it is C:\MY-EBOOKS\_Statistics that goes into the URL field for each of these books' records that I have in Zotero. Then, when I want to go from a Zotero record to the actual e-book file, I click on the URL in Zotero, and Firefox presents all the files in that C:\MY-EBOOKS\_Statistics folder as a webpage. If you have a certain naming scheme that you are following, for your files (e.g. Author-Title-Year-ISBN, or something like that), then it is easy to scroll down the list on the generated webpage, get to the file you wanted to open, and click on it. Basically, it takes two clicks to open the needed file: a click on the URL field in your Zotero record, and then another click on the webpage generated by Firefox.
One disadvantage here is that Zotero records have only one URL field (which is quite unfortunate, since it creates a problem under other scenarios as well). If you use up the URL field for storing your hard drive folder path, but still need some other web address added to the record, you don't have a "clickable" field to put it in. Still, you can add that other URL to the record by adding it as an attachment: right-click on the record line in the central panel, select Add Attachment, then Add Link to Current Page. Alternatively, if I know I won't need to go to that web address often, so I don't need it to be "clickable" that badly, I put the URL(s) in the Extra field of the record.
I can't say I am absolutely happy with this solution -- there is always the risk that some changes in the future versions of Zotero may make this workaround obsolete, or more difficult to use. But that's the risk you always face when coming up with your own shortcuts / macros / other ways of working *around* software features. Still, I am sticking to it for now, since I've invested too much time and effort into building my Zotero collection, and switching to Mendeley / Wizfolio / anything else would be more resource-intensive.
If you find this solution of use but have ideas on how to improve it further, I'd appreciate hearing from you.
There are a number of reasons why some people may not like moving their important content files out of their user directory, but it works for me... FWIW
I save all the documents in a fold in the zotero directory. I hope that I can copy them into a portable disk and access then by Zotero on any PC.
Also, it is helpful that the database can be accessed from different system with Zotero, such as Windows and Linux.
Moving things to the root directory is not an option for people in any sort of "corporate" environment, including at a university with a standardised PC setup.
Of course there are issues: mainly, that zotero only supports one url per document, and that copyrighted materials (such as proceeding files provided in conferences) are *theoretically* accessible to everyone (everyone knows the url..) this way!
I absolutely vote for relative path support.
Also, the crucial feature avoiding lock-in is the availability of good export options - Zotero offers all standard export formats as well as an open standard, which allows e.g. one click syncing to Mendeley.
Compared to data export, the file structure is just not a very important issue.
But I still agree 100% with adamsmith on the core issue-- Zotero tries to help data keep moving, keep being exchanged. The internal database structure and the file structure Zotero uses should not matter to you. Zotero will gladly export the files and data in a number of standard formats when you decide to do so. I suppose the important thing to remember is that Zotero is not designed as a file organizer. It is designed to be a flexible cross-platform research environment, so development focuses on ways to do that well.
Home-dir relative storage of linked files (the topic of this thread) would be a good improvement-- but the lack of this feature says _nothing_ about Zotero's philosophy.
In effect the use of directory names that are jumbled character strings means even though the files are intact, they are not stored in any logical way if you tried to navigate the tree from outside of the software. This is a negative in my book as it means if/when I want to stop using zotero I have an impossibly large re-organising task to re-acess those PDFs that in all honesty could never be accomplished. This is why my preference would be to store the files elsewhere in some other logical (to me) format (alphabetical or organsied by subject) and then link the whole lot into zotero. For references like web pages I can see where saving a snapshot into zotero on the fly is desireable. Nearly all my library is PDFs of scientific articles stored already in a format where I can access them independently to any software and linking them into zotero is the logical choice but with changes in storage, (e.g. local pc to fileserver, or adding a new drive) the option to move the storage tree's path without breaking the links is highly desirable.
In an earlier post you wrote that you are dependent on the money from sponsors and that they determine priorities of further developments. What's about donations? I think people would be quite willed to donate some money for a specific feature. Donating without knowing that the things most desired are implemented is not very encouraging.
One other comment. I really like the new Zotero 3.0b2.1 standalone and have decided to go ahead and make the switch from Endnote. In addition to the attached link storage model, it appears that this beta version allows only in-Zotero storage for drag-drop of pdfs/jpgs. Is there currently or a planned feature that drag-drop could default to attach by link by file vs attach/store copy of file (in Zotero)? Perhaps as a preference option?
I assume patches would be gladly accepted though.
Maybe of interest to others posting here: many people's needs may be served by Zotero plugins such as Zotfile and Zotfile reade, esp. wrt. syncing to tablets. Zotfile now also supports some version of relative paths.
I haven't tested everything yet and it may not work with platforms other than windows but it may give ideas on ways to work around the problem. My solution is to make your OS make the "relative path" by mapping a network drive.
Say your pdf library is in C:\documents and settings\user\my documents\references. Map the network drive (for exemple drive Y:) to be C:\documents and settings\user\my documents\references.
Then if you have xx.pdf in your library and you attach it to an element of your zotero library as Y:\xx.pdf and you make the same directory and network map drive on the other computer, the path always stay Y:\xx.pdf. Then if you make the mapped directory sync with dropbox or other web service and you sync your database through zotero syncing service, everything should work ok.
I just made up this solution this morning and haven't tested yet on a second computer. But with "show file" in zotero, it really point to Y:\xx.pdf and not the the whole path mapped by Y:.
(i) link files in your personal way and change the sqlite database to point to that files:
%path to zotero profile%\zotero.sqlite
- table "itemAttachment"
- variable "path" - it has the path to files, that files in zotero storage are named "storage:filename.pdf", files in your OS are named "C:\way\to\file\file.pdf"
- sync will not work to linked files, you have to sync on your own (dropbox?), but in each machine you have to put the files in same way.
- different OSes problem.
- cons: hardworking -> open sqlite db and change each folder.
(ii) you can change the dynamic named folders inside zotero storage.
- in zotero.sqlite, table "item" every attachment has a "key", this key is the path in storage directory, if you change the key and the folder to the same way, you may put more than one file per folder.
- sync will work because the files are inside storage folders.
- pros: you stay with your own selflogicnamed folders
- cons: hardworking -> open sqlite db and change each key.
(iii) the easy way: Install wampp, xampp, ....., "portable"
- run xampp portable
- create a symlink in htdocs inside xamp to point to your folder structure. Ex.: C:\xampp\htdocs\Reference <=> J:\Files\any\PDFs.
(http://www.howtogeek.com/howto/windows-vista/using-symlinks-in-windows-vista/)
- In Zotero, create a reference and "Attach link to url". Ex. http://localhost/Reference/filename.pdf
- Thats it.
- Pros: This solution can be used in any OS (ok, I didn't test any xampp solutions in linux either in mac, but I think it's possible). Your folder structure doesn't change, only symlinks changes...
- cons: lost pdf index.