PS - Just to confuse things further, Zot_data had the "Storage" subfolder created (alongside other 3) with the "randomly named" sub-sub-folders used for storing individual item data & attachments - a quick opening of some of them (about 10 - 20) indicated *.pdf files inside each. In other words, ZS was (somehow) identifying the correct attached files & ?copying them to the respective storage folders, but not making them available as "attachments" in the ZS interface. I would need to go thro each such storage folder & identify each item has correct pdf attached which in turn is correctly labelled (named) - I don't have time to do this at mo, but quick look at 4 or 5 suggests A MIX OF LOCATION between files stored in the Zot_final pdf folder (ZotFilePointer) & the Storage folders of ZS, namely Zot-data. Where the old ZotFile pointers are hiding I have no idea - remember I disabled ZotFile. A-HA : these items are downloaded from server which had data uploaded yesterday from PC1 with ZotFile working, therefore on PC2 with ZotFile diabled they still "remember" file location as Zot_final pdf.
I really don't get it cos 1000s of others must be happily doing this: 1) Using PC1, download/capture pdf & relevant metadata plus store & label item attachment; 2) Sync this item to server with attachment stored on server; 3) Open PC2 ZS at later date, & sync with server & hey presto, the recently captured item & attachment downlaoded onto PC2. Simple????
So the first thing I would ask you is, while troubleshooting a problem, only to do things when either a) Dan or I explicitly tell you to or b) you are 100% certain you understand exactly what they're doing and why you're performing them.
You're doing a lot of things that are not just a waste of your time, but also make it harder to troubleshoot for us and, in some cases, may even have made things worse (though I'm not sure about that). So don't feel the need to try out a lot of different things. Just let us walk you through the steps and report what you're seeing. Screenshots are great, too, where you feel that words are imprecise. If instructions are unclear, just ask. We'll try to respond as quickly as possible.
The reason you're running into trouble is that you've been using ZotFile without fully appreciating what it does. The moment you're using ZotFile's "File Location" function set to a custom folder, you're moving files out of Zotero and Zotero will no longer sync them. That's the root of all of your file sync issues. Zotero does all of the things you want it to do without any glitch and out of the box if you just let it do it's job--the reason you're running into problem are your attempts to customize its behavior without fulling understanding what your customizations are doing.
1. So, the first thing you want to do is go in to the Zotfile preferences, and under File Location, set it to "Attach Stored Copy of File". You can keep using ZotFile for renaming files if you like how the way it does that, so don't change anything else about ZotFile.
2. Now you need ZotFile to move the files back to Zotero's storage folder. For that purpose, select all your items in Zotero (let's do all of this on PC1)--i.e. click on "My Library" on the left and then in the middle panel, use ctrl+a, right-click -->Manage Attachments --> Rename Attachments
3. You will now, in Zotero, see that the icon for many of your attachments has changed from a link icon (chain link) to a PDF icon.
4. Now you want to sync these files to the server. Just run a regular sync at first. Spot check if some of the files are now available online (i.e. you can _open_, not just see them, on zotero.org). If they are continue to the next step. If (and only if) they're not, use "Reset File Sync History" in the Reset Menu of the Sync preferences. don't use any of the other options there.
5. Once you've confirmed the files are on the server, no sync PC2. I'd hope this should just work, but at this point there are a lot of moving parts, so we may have to revisit that last step.
6. Set both computers to sync automatically in the sync preferences to prevent sync conflicts from arising in the future.
OK Sir, point taken ;-)- sorry to be adding further to your workload!
I won't get around to following your instructions until a bit later, but could you clarify:
1) Should I (re)purge my library on server
2) Following your instructions, WHERE will ZS store the (renamed) pdfs (ie "move the files back to Zotero's storage folder.") Where is this - I guess wherever ZotPrefs->Advanced-> Files & Folders -> DataDirectory Location is pointing - in my case "C:\Zot_data", or given that ZotFile is still active, then its location is "C:\Zot_final_pdf.
reason I am asking for 2) above is a) so i can backup these *.pdfs as part of normal HDD backup; but b) I was told to keep these final pdfs separate from the Zotero work folder (ie C:\Zot_data).
(your regular HD backup should include the whole Zot_data folder, including the zotero database inside it as Dan says above).
I was told to keep these final pdfs separate from the Zotero work folder (ie C:\Zot_data).
that was either a misunderstanding or misinformation. (Most likely the former: this _is_ true if you want to use a service other than Zotero, like Dropbox, to sync PDFs)
OK - at risk of muddying water further: I want a folder/sub-folder/whatever with just the "correctly renamed pdfs" present, and not (as I suspect your suggestion will do) each pdf hidden in one of many storage folders. This is for a load of practical reasons including 1) simple backup of ALL my files - so not just Zotero - by eg OneDrive, SugarSync etc; & 2) I like to be able to just go into Windows File explorer & search for a pdf - perhaps I want to attach to an email; & 3) a lot of people I collab with are outside of "strict academia" & don't use Zotero (so sharing not an option).
Depends. The back-up might be tricky, but for
2) and 3)
use a virtual folder (naming depends on windows system):
http://en.wikipedia.org/wiki/Virtual_folder#Windows
for PDFs in Zot_data storage and place it anywhere on your computer (outside of Zot_Data, though).
Adam - thanks - steps 1 & 2 have now locked up my PC1, so I guess it will be tomorrow morning when I will get to see if it is working. In the meantime can you check the directories on
https://www.dropbox.com/s/97rbw7cfj4pif3b/1.PNG?dl=0
(the location of files is greyed out (bottom))
ZotFile settings are correct (location is supposed to be greyed out--it's handled by Zotero),
2nd question:
probably because some items have multiple attachments (I think the number may include snapshots and ZotFile will throw a message that those haven't been moved in the process).
It's certainly possible that some items have duplicate attachments--one way this happens is when merging duplicates--but it's generally not a major cause of concern. I don't think there's currently an easy way to find items with multiple attachments.
Well - not quite as smooth as I hoped - Your steps 1 - 3 above locked up my laptop (PC1) (an i7 Sandybridge & 8Gb RAM) for 16+ hours. When completed, there were 200 odd items (out of 4550) without attachments. What was VERY odd was that the old Zot_final_pdf where many of the attachments resided had reduced from 3767 files to 213?? However, I have not started chasing down the orphans (manually). I am now syncing with server - tempted to say initial results don't look good (the 20 odd I opened at random did not have attached pdfs) but I guess this may run over the w/e
One thought about getting a folder with just my pdfs sitting in it all nicely renamed is to retain Zotfile General Settings as in yesterdays *.png, but then use (?manually) Zotfile to send the pdfs to a folder for tablet downlaod - its just my tablet is a hard drive on my laptop? Would that work? ((I've searched the web & Z fora - seems to be an ongoing request without a clear answer)
Re duplicates - yup - a pain - I tend to search every month or so using dupeGuru, which was fine when my attachments were set to sit in C:\Zot_final_pdf - it did mean I then had to manually go back & locate when the "wrong" copy was deleted ...
One Q - what happens to the multiple attachments under "merge" option in ZS Duplicates?
Currently multiple attachments under merge duplicates just become, well, duplicate attachments. The concern is that we really don't want to auto-delete a marked-up copy of a file. We're looking at improving the algorithm for this (e.g. by comparing hash sums) but no ETA on that.
The send to tablet function will work to create a copy of PDFs in a folder on your HD, but it won't do so automatically, i.e. you have to "send to tablet" manually (though you can do that for a batch of files).
UPDATE: Hi - well - not a rip-roaring success!! In fact, I have had to stop syncing, cos this seems to "delete" attachments from my HDD. Really not sure what is going on - as I carried out the agreed protocol as above - took a while for all the upload/download. But on PC1 & PC2 kept getting (differing) parent items being shown as lacking attachment. However, when I went in to each (well, some) of those lacking an attachment, I found another eg 4 child pdfs attached, plus one registering as "missing" (open circle) - so why is ZS not picking up the other 4 present? Also, which item flags up a missing child seems slightly arbitary - eg this afternoon I went thro PC1 & manually deleted obvious duplicate (parent level + associated attachments) items, & at end of it, sorted on "attachment", & those missing all started between P & T (I was working alphabetically). Might be coincidence ...
But syncing really seems to throw spanner in works - so i am currently getting ZS on PC1 "working properly" & all files present & correct, & then I will delete everything on server & resync (but keep PC2 isolated from server). I will then add a few files to ZS on PC1 & see if that behaves properly, before trying to sort PC2 ...
By the way - files (attachments) now uploading to server (well, those I randomly checked) & Zotfile is sending off my pdfs to a dedicated folder on HDD - so getting there - slowly!
OK – things REALLY not going well!! Decided the only way to clean my Zot files was to open view all attachments & go thro file by file. Took me better part of 6 hours, then went to Trash & deleted, & then “Tablet Files” & hit removal all (delete). Not clever, because of course when one clears a “Collection” one clears them from My Library. BIG RED WARNING SIGNS FOR IDIOTS LIKE ME PLEASE – it is too easy to think they are equivalent to sub-directories (folders) in normal Tree directory structure.
So back to square one – I had luckily (thro Zotfile) exported every pdf (including all the duplicates) to a separate folder, & I had back-up from my “clean” version last night – this had everything properly attached, plus MANY duplicate attachments – the ones I had manually deleted this morning! So I have removed all duplicates from the “pdf folder” & I am not copying all these pdfs back into ZotStandalone, NOT connected to the server, & one day someday I will extract all the metadata again. Objective – a lean clean Zotfile folder lacking duplicates & that syncs seamlessly with server & other PCs. I doubt it, but one lives in hope.
So some feature (/future) requests:
1) Is it possible to have a “progress bar” rather than simply a spinning wheel & “Not Responding”. I don’t want it to the nearest second – just should I make a coffee, use the other PC, go to the pub or come back in the morning.
2) Many of my duplicates arise from browsing one paper on the web, & coming across a number of other “interesting/relevant” papers – 30 years ago when I did my DPhil the litt was such I could probably quote all relevant papers verbatim – no more – there is so much litt out there (mainly crap) that I cannot keep on top of it – hence tools like Zot. S would it not be possible to flag a potential duplicate download before physically downloading file into Zot. You do enough clever things in zot, I would have thought it not too difficult (at least for papers & other stuff with unique identifiers (eg DOI)).
3) I know you guys do sterling work especially when people like me apparently intentionally go out & make life difficult. Would not a fairly extensive Idiots Guide to Zot actually save you time? Yes – i do use the discussions & the Zot Forum (& the more general web), but my search query still just shows a wheel going around & around (something I raised a while back – perhaps because I use Ghostery / ABE & others?)
Anyway – thanks for all your help – when I FINALLY get a clean version working on my HDD to my satisfaction, I will try syncing with server – I may be back asking for help then!!
PS - and
4) A "Permanently Delete All" button
5) A delete all button for the library
6) a RESTORE button ;-) for when EEJITS like me accidentally delete a Collection (or part thereof) - those deleted are often difficult to find in the Trash
7) so also, Date & time of deleted items in trash
Fair exchange for the 100 quid i probably won't be using ;-) [[I somehow doubt I will be syncing with server unless its seamless - I prefer the old m,anual way - at least I know it works!!]]
Upgrade Storage
I really don't get it cos 1000s of others must be happily doing this: 1) Using PC1, download/capture pdf & relevant metadata plus store & label item attachment; 2) Sync this item to server with attachment stored on server; 3) Open PC2 ZS at later date, & sync with server & hey presto, the recently captured item & attachment downlaoded onto PC2. Simple????
a) Dan or I explicitly tell you to or
b) you are 100% certain you understand exactly what they're doing and why you're performing them.
You're doing a lot of things that are not just a waste of your time, but also make it harder to troubleshoot for us and, in some cases, may even have made things worse (though I'm not sure about that). So don't feel the need to try out a lot of different things. Just let us walk you through the steps and report what you're seeing. Screenshots are great, too, where you feel that words are imprecise. If instructions are unclear, just ask. We'll try to respond as quickly as possible.
The reason you're running into trouble is that you've been using ZotFile without fully appreciating what it does. The moment you're using ZotFile's "File Location" function set to a custom folder, you're moving files out of Zotero and Zotero will no longer sync them. That's the root of all of your file sync issues. Zotero does all of the things you want it to do without any glitch and out of the box if you just let it do it's job--the reason you're running into problem are your attempts to customize its behavior without fulling understanding what your customizations are doing.
1. So, the first thing you want to do is go in to the Zotfile preferences, and under File Location, set it to "Attach Stored Copy of File". You can keep using ZotFile for renaming files if you like how the way it does that, so don't change anything else about ZotFile.
2. Now you need ZotFile to move the files back to Zotero's storage folder. For that purpose, select all your items in Zotero (let's do all of this on PC1)--i.e. click on "My Library" on the left and then in the middle panel, use ctrl+a, right-click -->Manage Attachments --> Rename Attachments
3. You will now, in Zotero, see that the icon for many of your attachments has changed from a link icon (chain link) to a PDF icon.
4. Now you want to sync these files to the server. Just run a regular sync at first. Spot check if some of the files are now available online (i.e. you can _open_, not just see them, on zotero.org). If they are continue to the next step. If (and only if) they're not, use "Reset File Sync History" in the Reset Menu of the Sync preferences. don't use any of the other options there.
5. Once you've confirmed the files are on the server, no sync PC2. I'd hope this should just work, but at this point there are a lot of moving parts, so we may have to revisit that last step.
6. Set both computers to sync automatically in the sync preferences to prevent sync conflicts from arising in the future.
I won't get around to following your instructions until a bit later, but could you clarify:
1) Should I (re)purge my library on server
2) Following your instructions, WHERE will ZS store the (renamed) pdfs (ie "move the files back to Zotero's storage folder.") Where is this - I guess wherever ZotPrefs->Advanced-> Files & Folders -> DataDirectory Location is pointing - in my case "C:\Zot_data", or given that ZotFile is still active, then its location is "C:\Zot_final_pdf.
reason I am asking for 2) above is a) so i can backup these *.pdfs as part of normal HDD backup; but b) I was told to keep these final pdfs separate from the Zotero work folder (ie C:\Zot_data).
Thanks
R
2) C:\Zot_data/storage
(your regular HD backup should include the whole Zot_data folder, including the zotero database inside it as Dan says above). that was either a misunderstanding or misinformation. (Most likely the former: this _is_ true if you want to use a service other than Zotero, like Dropbox, to sync PDFs)
Am I asking the impossible?
2) and 3)
use a virtual folder (naming depends on windows system):
http://en.wikipedia.org/wiki/Virtual_folder#Windows
for PDFs in Zot_data storage and place it anywhere on your computer (outside of Zot_Data, though).
https://www.dropbox.com/s/97rbw7cfj4pif3b/1.PNG?dl=0
(the location of files is greyed out (bottom))
& why 4551 items selected but 4862 attachments - presumably duplicates & deleted items?
https://www.dropbox.com/s/hjvsaj71y875u80/2.PNG?dl=0
Thanks
R
2nd question:
probably because some items have multiple attachments (I think the number may include snapshots and ZotFile will throw a message that those haven't been moved in the process).
It's certainly possible that some items have duplicate attachments--one way this happens is when merging duplicates--but it's generally not a major cause of concern. I don't think there's currently an easy way to find items with multiple attachments.
One thought about getting a folder with just my pdfs sitting in it all nicely renamed is to retain Zotfile General Settings as in yesterdays *.png, but then use (?manually) Zotfile to send the pdfs to a folder for tablet downlaod - its just my tablet is a hard drive on my laptop? Would that work? ((I've searched the web & Z fora - seems to be an ongoing request without a clear answer)
Re duplicates - yup - a pain - I tend to search every month or so using dupeGuru, which was fine when my attachments were set to sit in C:\Zot_final_pdf - it did mean I then had to manually go back & locate when the "wrong" copy was deleted ...
One Q - what happens to the multiple attachments under "merge" option in ZS Duplicates?
Many thanks
R
The send to tablet function will work to create a copy of PDFs in a folder on your HD, but it won't do so automatically, i.e. you have to "send to tablet" manually (though you can do that for a batch of files).
Cheers
R
But syncing really seems to throw spanner in works - so i am currently getting ZS on PC1 "working properly" & all files present & correct, & then I will delete everything on server & resync (but keep PC2 isolated from server). I will then add a few files to ZS on PC1 & see if that behaves properly, before trying to sort PC2 ...
By the way - files (attachments) now uploading to server (well, those I randomly checked) & Zotfile is sending off my pdfs to a dedicated folder on HDD - so getting there - slowly!
Will keep you posted
R
So back to square one – I had luckily (thro Zotfile) exported every pdf (including all the duplicates) to a separate folder, & I had back-up from my “clean” version last night – this had everything properly attached, plus MANY duplicate attachments – the ones I had manually deleted this morning! So I have removed all duplicates from the “pdf folder” & I am not copying all these pdfs back into ZotStandalone, NOT connected to the server, & one day someday I will extract all the metadata again. Objective – a lean clean Zotfile folder lacking duplicates & that syncs seamlessly with server & other PCs. I doubt it, but one lives in hope.
So some feature (/future) requests:
1) Is it possible to have a “progress bar” rather than simply a spinning wheel & “Not Responding”. I don’t want it to the nearest second – just should I make a coffee, use the other PC, go to the pub or come back in the morning.
2) Many of my duplicates arise from browsing one paper on the web, & coming across a number of other “interesting/relevant” papers – 30 years ago when I did my DPhil the litt was such I could probably quote all relevant papers verbatim – no more – there is so much litt out there (mainly crap) that I cannot keep on top of it – hence tools like Zot. S would it not be possible to flag a potential duplicate download before physically downloading file into Zot. You do enough clever things in zot, I would have thought it not too difficult (at least for papers & other stuff with unique identifiers (eg DOI)).
3) I know you guys do sterling work especially when people like me apparently intentionally go out & make life difficult. Would not a fairly extensive Idiots Guide to Zot actually save you time? Yes – i do use the discussions & the Zot Forum (& the more general web), but my search query still just shows a wheel going around & around (something I raised a while back – perhaps because I use Ghostery / ABE & others?)
Anyway – thanks for all your help – when I FINALLY get a clean version working on my HDD to my satisfaction, I will try syncing with server – I may be back asking for help then!!
Cheers
R
4) A "Permanently Delete All" button
5) A delete all button for the library
6) a RESTORE button ;-) for when EEJITS like me accidentally delete a Collection (or part thereof) - those deleted are often difficult to find in the Trash
7) so also, Date & time of deleted items in trash
Fair exchange for the 100 quid i probably won't be using ;-) [[I somehow doubt I will be syncing with server unless its seamless - I prefer the old m,anual way - at least I know it works!!]]
R