Problems dowloading files and entries from a database

Canadian photographi
Hi,

This problem is probably related to a previous problem that I thought I'd resolved so let me go back to the beginning.

I had to hard wipe and reinstall my IMAC recently and, without realizing it, I was using zotero without having brought over my data. The sync funtion was on and so zotero had loaded everything in from that back up but, of course, I was missing my files.

So long story short, today I finally brought over my original zotero and custom linked it in. I also turned off syncing for my personal library because I think it will be easier in future. My group library syncing is still on.

I had a couple problems attaching my data directory (hence having to use the custom data directory funtion) but it all seemed to be working, ie. I have my files back! But now, when I try to save an entry I'm getting serious freezing.

There are no problems with entries from my university library or the web, only with pdf attachments and entries from a database. I can't even drop in a file and attach it to an existing entry.

I suspect this has something to do with how I attached up the data directory and storage through custom.

Can you help?

Thanks,
karla
  • Canadian photographi
    I'm also having a syncing problem now! Sorry, this is turning into a mess. My report ID is: 120668465
  • aurimas
    Dan will take a look at the Error Report, but you'll probably want to explain in detail what you did to "custom link" your original Zotero library.
  • dstillman Zotero Team
    There are no Zotero errors in that report. We'll want an answer to aurimas's question — there's no reason you should have to use a custom data directory unless you actually want to.

    Can you provide a Debug ID for a freeze (assuming it's temporary and you don't need to restart Firefox)?

    Also, on this:
    The sync funtion was on and so zotero had loaded everything in from that back up but, of course, I was missing my files.
    That "of course" was sarcastic, right? Because if you were using file sync you should certainly have all your files back after syncing — and you have multiple gigabytes of files online. See Files Not Syncing for the general troubleshooting steps, though we should look at the freezing first.
  • Canadian photographi
    Hi all, thanks for all your replies.

    1. By "custom link" I mean that I went into preferences and, under data directory location I chose custom and then found the "zotero" folder with all my squite and storage stuff.
    2. I used custom link because I couldn't get zotero to recognize my folder, it just kept creating a new zotero folder rather than hooking up to the one I wanted, ie. with all my files.
    3. I opened and closed firefox multiple times and also restarted my computer a bunch of times when trying to hook up the proper zotero library folder so it isn't a temporary problem.
    Here's the Debug ID for the problem I'm having saving entries and pdfs from a database (ebsco host) D1316261974.
    4. I am not being sarcastic actually! I spend several hours reviewing the provided directions for sync files to get a sense of why my files had disappeared. My understanding is that files that are part of a group library are stored on the online zotero (which I pay for) but that my personal library attachments are found on my home computer. All my group library attachments were still on the online zotero storage but my personal stuff wasn't so I took this as confirmation. Especially as, once I reinstalled and entered my personal id in settings, zotero created a new data directory without the original My Library files. THe instructions provided by you guys on the Files Not Syncing page are great but not applicable to me as I only use on computer. Hence the whole if computer A then computer B thing doesn't really help me out. I also tried the whole Reset File Sync History thing, thinking this would resolve things but nothing changed.
    5. THe syncing problem says "transaction in progress in Zotero.Sync.server._check sync user." Then it asked me to submit a report to zotero, report id 1087939196. Is this because I unchecked "sync attached files in my library using zotero"? I did this just to avoid the same problem of having my personal attached files disappear again.

    Thanks, karla
  • Canadian photographi
    And just to clarify this comment:

    "The sync function was on and so zotero had loaded everything in from that back up but, of course, I was missing my files."

    I suspect that because I had reinstalled zotero but had not yet moved and attached my previous data directory, zotero just made a new one from zotero.org but without my attached files in My Library. I assume that this happened because zotero synced from the zotero.org site to my data directory before I realized would happen. Doesn't that make sense? I have to say I was surprised but ... . The only other thought was that my data allotment filled up and my personal library was axed without my knowledge. Could that have happened?
  • aurimas
    edited July 23, 2014
    The only other thought was that my data allotment filled up and my personal library was axed without my knowledge. Could that have happened?
    Yes, but you could check this online and it would have given you a sync warning. Also, this would have prevented syncing files to any group libraries that _you_ own.

    We likely won't figure out why your files were not syncing from your other computer, but we can make sure that they sync now once your new setup is in order.

    Now, as for your current setup. Dan will have to take a look at your Debug ID to see what errors you are getting, but if I had to venture a guess as to what's happening, I think you may be running into file permission issues. I'm not a Mac user, so I won't trust myself to give you detailed instructions, but you need to make sure that the Zotero data directory _and_ all subdirectories that you copied from your other computer are _owned_ by your current user _and_ that you have read/write permission to every directory and file inside. Typically, you can do this on a POSIX system using chown command with an -R flag followed by a chmod command with the -R flag (obviously you have to specify your username and group). There might be a way to do it through the UI (idk).

    Also, I assume that you copied the data directory somewhere to your hard disk and it's not anywhere on a removable media, network drive, or something like a Dropbox folder.
  • Canadian photographi
    Thanks aurimas.

    My husband and I were also thinking that permissions might be an issue. This is a little over my technical pay-grade but I'm going to have him take a look at your suggestions and see if we can solve it.

    Yes, my data directory is on my hard disk!

    I may delete the copy of my data directory that I'm trying to use and reinstall it just to start fresh (I've a copy in my admin user). Is there any particular protocol for doing this?

    Also, would you suggest—if I get it working—that I try and "restore to zotero server" afterwards? Of should I just keep my library offline?
  • aurimas
    I may delete the copy of my data directory that I'm trying to use and reinstall it just to start fresh (I've a copy in my admin user). Is there any particular protocol for doing this?
    Unless you did something weird after copying it the first time, this is unlikely to help.
    Also, would you suggest—if I get it working—that I try and "restore to zotero server" afterwards?
    No, unless there's a good reason to do this (and there's currently not), you should not restore to or from server.
    Of should I just keep my library offline?
    There's also no reason to do this. Simply syncing your files should take care of everything once you resolve your data directory issues.
  • dstillman Zotero Team
    You can edit permissions via the Get Info dialog for a folder in Finder.
    THe instructions provided by you guys on the Files Not Syncing page are great but not applicable to me as I only use on computer.
    No, they're still applicable. The main point in your case would be to check the files online to see if they're accessible there. You currently have over 600MB of files in your personal library — nearly 700 files. But it looks like only a handful of them were created in the last year and a half. So it's likely not that any of your files were wiped out but that you simply hit your file quota at some point and, perhaps, turned off syncing in your personal library because of it. (It looks like files have been syncing to your group library this whole time.) As aurimas says, you would've gotten a message when you hit your file sync quota.

    As for your Debug ID, there does appear to be a problem, but no error was logged for some reason. Can you disable your other Firefox extensions and submit another for a another save attempt?
  • dstillman Zotero Team
    edited July 24, 2014
    And as aurimas says, don't do anything with your data directory or reinstalling or resetting or anything like that at this point (other than checking permissions in Get Info to make sure your active data directory and all files within are owned and writeable by your user account).
  • Canadian photographi
    Hi aurimas and Dan,

    Here's a new debug id: D496371970

    I only had one other firefox extension besides zotero.

    I corrected all the permissions discrepancies in my data directory and now I can import from a database but I'm still having the syncing issues.

    I can't seem to access any of the files I have in my library from zotero.org. As far as I know I've never received any emails telling my storage quota is filled or changed the syncing function of my library.

    thanks, karla
  • dstillman Zotero Team
    Well, again, many of your personal files do appear to be unsynced, but you did have 600MB of them on the server yesterday, so you should have been able to access those. For example, the PDF attachment to this item should be accessible:

    https://www.zotero.org/canadian_photographic_history/items/itemKey/UGSNQ2KU

    You now have 1500MB of files synced in your personal library, I assume from a recent upload, but most of your recent files still haven't synced. I can't tell you why those never synced, but most likely you had turned off file syncing in your personal library for some reason. I'm afraid there's nothing we can do for those if you don't have a backup of your Zotero data directory from the old computer.
  • aurimas
    As far as I know I've never received any emails telling my storage quota is filled
    This would have showed up in the Zotero client (not via email), but it's possible that you didn't notice it.

    Are you getting any sync errors? (look for a red exclamation mark next to a green circular sync arrow in the top right corner of Zotero)
  • Canadian photographi
    Hi aurimas,

    Yes, I am getting a red exclamation mark syncing error symbol. Is that not what the above debug id is about?

    If not, my report error is 946540895 for the syncing issue.

    Hi Dan,

    Yes I do see that item you chose but this item is not accessible:

    https://www.zotero.org/canadian_photographic_history/items/itemKey/GVT3UPNW/q/giannachi

    This is one I just added today and can access through my personal library just fine but not on the server.

    I do have a back up of my personal library, this is what I've been saying. That is why I'm confused as to why some items are not found on the server, if they are supposed to be backing up. Since the pdfs are in my data directory on my computer should they not have synced to the server more recently (as my file syncing is now on)?

    I've been using zotero since 2009 so the fact I have 1.5 gb of material isn't surprising. It is true that since I repaired permissions the amount increased from 1.2 to 1.5 gb but I don't understand why it is still having syncing issues. Do I not have enough storage? IF I take my personal library offline, will the syncing errors be resolved?

    karla
  • dstillman Zotero Team
    Yes, I am getting a red exclamation mark syncing error symbol. Is that not what the above debug id is about?
    The Debug ID isn't sufficient — there's lots of info in the debug output, but it's important to tell us specifically when you're getting a sync error. Note that the Files Not Syncing page I linked above mentions file sync errors as a reason files wouldn't sync.

    But is this actually a persistent error that you get every time, or did you just see the icon once? If you press the sync button again, does it keep happening?
    Yes I do see that item you chose but this item is not accessible:

    https://www.zotero.org/canadian_photographic_history/items/itemKey/GVT3UPNW/q/giannachi

    This is one I just added today and can access through my personal library just fine but not on the server.
    Right, but then that's a separate issue, caused by the sync error you're currently getting, and not related to the couple years of unsynced files (unless you were getting file sync errors for a couple years, which is a possibility if you weren't looking for them).
  • dstillman Zotero Team
    It looks like you just got that single error and then stopped trying to sync. To be clear, intermittent file sync errors aren't uncommon and are nothing to worry about. You can just click the sync icon again or continue to use Zotero normally with auto-sync enabled and let it continue syncing on its own.

    So if you just sync again your files should continue to sync — though this will only apply to the files you just added to your current computer. We can't help you with files that didn't sync previously and you don't have a backup of.
  • aurimas
    Could you also check how many folders (don't include subfolders) you have in the "storage" folder inside your Zotero data directory (both in your backup of the old Zotero data directory and the one you are currently using)?
  • Canadian photographi
    Hi Dan,

    Actually, I've been trying to sync all day and getting the persistent syncing error. This has been happening ever since I reinstalled my backed up data directory (ie. before I contacted you).

    At first I thought the problem was going to be fixed when I fixed the file permissions but it wasn't.

    I have not been having file sync errors for the last couple of years. I would have definitely noticed and taken steps to fix it!

    Just to be clear, I do have a back up of all my files that are not appearing on zotero.org. This is why I custom linked to my old data directory ... which led to the current issues. I can see them in my personal library as we speak (and open them) but when I check the online version they are absent. The entries on zotero.org list them as attachments but don't actually provide a link to them.

    So, perhaps, as you say "if you just synch again your files should continue to sync". Except for the fact I am having syncing errors ...

    Aurimas,

    I have a load of folders in my "storage" file in my zotero data directory.
  • Canadian photographi
    I will get back to with the exact numbers shortly!
  • Canadian photographi
    Okay, the newly reinstalled zotero directory: 155,727 items

    The older one which I am no longer using: 64,175 items
  • aurimas
    I can see them in my personal library as we speak (and open them) but when I check the online version they are absent. The entries on zotero.org list them as attachments but don't actually provide a link to them.
    I see. Not sure about Dan, but my understanding was that you are not able to open them locally. If you can access all of you files from your local computer, then we just need to help you fix the sync error and everything should be in order.
  • Canadian photographi
    Hi Aurimas,

    Sorry for the confusion, yes I can open them locally. I have been able to ever since I custom linked up the backed up library.

    Yes, hopefully the syncing problem would solve everything!
  • dstillman Zotero Team
    OK, so then let's start again (though the Files Not Syncing page was still the relevant documentation for this). Can you restart Firefox and provide another Debug ID for a file sync that fails?
  • dstillman Zotero Team
    edited July 25, 2014
    Well, except I still think these are just intermittent errors, which sometimes happen with very large initial syncs. Every time I check your account you have several hundred more MB in your personal library — you're currently up to 1800MB. So forget the Debug ID and just keep syncing. You can hover over the blue progress bar to see how many files it's syncing each time. You only need to report anything to us if it's failing immediately at the beginning of every file sync, but I think you'll find that's not the case.
  • Canadian photographi
    Hi again,


    Like I said, I've been syncing for the last two days and it always eventually fails (I'm not watching it continuously so I cannot guarantee the time of fail). Yes, I see the progress bar is moving slowly, there are a lot of files, but it always fails at some point.

    Dan, I've reviewed the files not syncing page multiple times.

    In step two, it says "If there's a sync error, indicated by an error icon to the left of the green sync icon in the Zotero toolbar, files may not be transferred, and you'll need to resolve the sync error before file syncing can resume. Click the sync error icon to view the message and, if necessary, post the error to the forums."

    This is what I'm doing. I don't see how the page itself could help me at this point.

    I will try to keep syncing and hope for the best. I restarted the sync this morning after last night's last fail. The debug ID above was produced after the fail. I at no point did I "stop" the sync manually.

    Is there some way to produce a debug ID on a singular failed file? I don't know how to do that.

    Karla
  • Canadian photographi
    Hi Dan and aurimas,

    So my system finally stopped syncing - and gave me the message that my storage quota was reached.

    That explains a lot.

    Thanks for all your help,
    Karla
  • dstillman Zotero Team
    Like I said, I've been syncing for the last two days and it always eventually fails
    That's not what I said though — I said it'd be a concern if it were failing immediately at the beginning of every sync, which wasn't the case for you. When syncing multiple gigabytes, intermittent errors should be expected and don't indicate a real problem. (We've made Zotero retry many such errors automatically in recent versions, but there are still some failures that aren't automatically retried. Later versions will likely be better about that.)

    Anyhow, yes, you are currently at your file sync quota, which doesn't have anything to do with the previous intermittent errors but will prevent you from syncing further.

This is an old discussion that has not been active in a long time. Before commenting here, you should strongly consider starting a new discussion instead. If you think the content of this discussion is still relevant, you can link to it from your new discussion.

Sign In or Register to comment.