[MacOs] Pdf attachment within "Storage" folder

olivierroymd

Hello everyone,

I used to keep all my Zotero PDF attachments in Google Drive using Zotfile but switched a couple of months ago to the Zotero storage to sync with their cloud service (unlimited, which I pay for).

The problem is that since then, I cannot find pdfs (which are located locally on my mac in /System/Volumes/Data/Users/myUSERNAME/Zotero/storage/PSEUDORANDOM/MYPDF.pdf) using Alfred/Raycast/Finder file search functions. However, I can find it using Spotlight.

The only workaround I found is to first open the pdf with Preview/PDF expert. Once it is done, the file can be found using previously mentioned means. The thing is, since Zotero Beta has implemented their own PDF reader and I use it instead of others; therefore I never have to open PDF in Preview/PDF expert and doing the workaround would require big changes in my workflow.

I already tested with Alfred's metadata tester and all is fine.

Any idea what's going on?
Thank you
  • dstillman
    This wouldn't be a Zotero question — they're just PDFs in a subfolder of your home folder, which macOS obviously should and usually can find. But in my experience Finder search is incredibly buggy, with bizarre behavior like results disappearing as soon as I open them (which actually sounds like the exact inverse of what you're seeing). I'm not sure if I've reproduced that in Big Sur, so if you're not on Big Sur, you can try upgrading to that. Otherwise, Monterey might improve things for you.

    You can also try rebuilding the Spotlight index, which would be the general advice you'd get from Apple.

    If that doesn't help and you do report this to Apple, you shouldn't mention Zotero at all, since this is just about files on disk, not about Zotero.
  • tcrow86
    FWIW I have this problem too. Running Big Sur - can't remember if it happened on Mac OS 10, but I think not because I only noticed the problem ~6mo ago despite regularly using Alfred for search. As far as I can tell, it's only happening with the PDFs in Zotero's storage folder, but only with some of them. If I actually open the file, that file will then immediately begin appearing as expected in Alfred's file search. This all despite all files appearing correctly under the 'PDF Documents' heading in Spotlight search. Rebuilding the index at as deep a level as possible seems to have no effect.

    Could well be a Spotlight/file system issue, but I thought it may be useful to post this here in case anyone has come up with a solution. Will post back if I notice that Monterey resolves the issue.
  • dstillman
    edited July 13, 2021
    Could well be a Spotlight/file system issue, but I thought it may be useful to post this here in case anyone has come up with a solution.
    It is absolutely a Spotlight issue (or an Alfred issue if it's only in Alfred — I don't know whether Alfred uses its own index like LaunchBar or relies on the system). This has nothing to do with Zotero. They're just files on disk in a normal folder.

    If it's only happening in Alfred, you should reach out to them for debugging.
  • dstillman
    edited July 13, 2021
    (If I had to guess, I'd say it's probably related to the 'quarantine' attribute, which has contributed to Spotlight bugs in the past. But that's something Alfred or Apple folks would have to help you with.)
  • olivierroymd
    dstillman, is it just me or you're overly defensive about this..?

    I'd like to point out something. I realized that the attribute kMDItemLastUsedDate IS NOT SET by Zotero when this pipeline is used: Saving reference from PubMed -> Go on to download the paper in pdf -> Add attachment from stored copy of file (-> automatically renamed from Zotero metadata with Zotfile).

    A quick Google search seems to relate this issue to individual apps responsible of "putting the file there" in the file system. Therefore, it would seem, it looks like a Zotero issue and the ability to set this metadata attribute should maybe be built in?

    Example:

    METADATA BEFORE OPENING THE FILE:

    XXXX:~ XXXX$ mdls /Users/XXXX/Zotero/storage/2AWD8N3T/RedickT_2014_Acta_Psychologica.pdf
    _kMDItemDisplayNameWithExtensions = "RedickT_2014_Acta_Psychologica.pdf"
    kMDItemAuthors = (
    "Thomas S. Redick"
    )
    kMDItemContentCreationDate = 2021-05-02 00:44:50 +0000
    kMDItemContentCreationDate_Ranking = 2021-05-02 00:00:00 +0000
    kMDItemContentModificationDate = 2021-07-10 09:00:00 +0000
    kMDItemContentModificationDate_Ranking = 2021-07-10 00:00:00 +0000
    kMDItemContentType = "com.adobe.pdf"
    kMDItemContentTypeTree = (
    "com.adobe.pdf",
    "public.data",
    "public.item",
    "public.composite-content",
    "public.content"
    )
    kMDItemCreator = "Elsevier"
    kMDItemDateAdded = 2021-05-02 00:44:50 +0000
    kMDItemDateAdded_Ranking = 2021-05-02 00:00:00 +0000
    kMDItemDescription = "ACTPSY, 145 (2014) 1-9. doi:10.1016/j.actpsy.2013.10.010"
    kMDItemDisplayName = "RedickT_2014_Acta_Psychologica.pdf"
    kMDItemDocumentIdentifier = 0
    kMDItemEncodingApplications = (
    "Acrobat Distiller 10.0.0 (Windows)"
    )
    kMDItemFinderComment = "zotero://select/library/items/7VQLPTEG"
    kMDItemFSContentChangeDate = 2021-07-10 09:00:00 +0000
    kMDItemFSCreationDate = 2021-05-02 00:44:50 +0000
    kMDItemFSCreatorCode = ""
    kMDItemFSFinderFlags = 0
    kMDItemFSHasCustomIcon = (null)
    kMDItemFSInvisible = 0
    kMDItemFSIsExtensionHidden = 0
    kMDItemFSIsStationery = (null)
    kMDItemFSLabel = 0
    kMDItemFSName = "RedickT_2014_Acta_Psychologica.pdf"
    kMDItemFSNodeCount = (null)
    kMDItemFSOwnerGroupID = 20
    kMDItemFSOwnerUserID = 501
    kMDItemFSSize = 438698
    kMDItemFSTypeCode = ""
    kMDItemInterestingDate_Ranking = 2021-07-13 00:00:00 +0000
    kMDItemKeywords = (
    "Cognitive control; Working memory; Individual differences"
    )
    kMDItemKind = "Document PDF"
    kMDItemLogicalSize = 438698
    kMDItemNumberOfPages = 9
    kMDItemPageHeight = 793.701
    kMDItemPageWidth = 595.2760009765625
    kMDItemPhysicalSize = 442368
    kMDItemSecurityMethod = "None"
    kMDItemTitle = "Cognitive control in context: Working memory capacity and proactive control"
    kMDItemVersion = "1.7"

    METADATA AFTER OPENING IN PDF EXPERT (therefore subsequently allowing to find in Alfred):

    XXXX:~ XXXX$ mdls /Users/XXXX/Zotero/storage/2AWD8N3T/RedickT_2014_Acta_Psychologica.pdf
    _kMDItemDisplayNameWithExtensions = "RedickT_2014_Acta_Psychologica.pdf"
    kMDItemAuthors = (
    "Thomas S. Redick"
    )
    kMDItemContentCreationDate = 2021-05-02 00:44:50 +0000
    kMDItemContentCreationDate_Ranking = 2021-05-02 00:00:00 +0000
    kMDItemContentModificationDate = 2021-07-10 09:00:00 +0000
    kMDItemContentModificationDate_Ranking = 2021-07-10 00:00:00 +0000
    kMDItemContentType = "com.adobe.pdf"
    kMDItemContentTypeTree = (
    "com.adobe.pdf",
    "public.data",
    "public.item",
    "public.composite-content",
    "public.content"
    )
    kMDItemCreator = "Elsevier"
    kMDItemDateAdded = 2021-05-02 00:44:50 +0000
    kMDItemDateAdded_Ranking = 2021-05-02 00:00:00 +0000
    kMDItemDescription = "ACTPSY, 145 (2014) 1-9. doi:10.1016/j.actpsy.2013.10.010"
    kMDItemDisplayName = "RedickT_2014_Acta_Psychologica.pdf"
    kMDItemDocumentIdentifier = 0
    kMDItemEncodingApplications = (
    "Acrobat Distiller 10.0.0 (Windows)"
    )
    kMDItemFinderComment = "zotero://select/library/items/7VQLPTEG"
    kMDItemFSContentChangeDate = 2021-07-10 09:00:00 +0000
    kMDItemFSCreationDate = 2021-05-02 00:44:50 +0000
    kMDItemFSCreatorCode = ""
    kMDItemFSFinderFlags = 0
    kMDItemFSHasCustomIcon = (null)
    kMDItemFSInvisible = 0
    kMDItemFSIsExtensionHidden = 0
    kMDItemFSIsStationery = (null)
    kMDItemFSLabel = 0
    kMDItemFSName = "RedickT_2014_Acta_Psychologica.pdf"
    kMDItemFSNodeCount = (null)
    kMDItemFSOwnerGroupID = 20
    kMDItemFSOwnerUserID = 501
    kMDItemFSSize = 438698
    kMDItemFSTypeCode = ""
    kMDItemInterestingDate_Ranking = 2021-07-13 00:00:00 +0000
    kMDItemKeywords = (
    "Cognitive control; Working memory; Individual differences"
    )
    kMDItemKind = "Document PDF"
    kMDItemLastUsedDate = 2021-07-13 14:19:49 +0000
    kMDItemLastUsedDate_Ranking = 2021-07-13 00:00:00 +0000
    kMDItemLogicalSize = 438698
    kMDItemNumberOfPages = 9
    kMDItemPageHeight = 793.701
    kMDItemPageWidth = 595.2760009765625
    kMDItemPhysicalSize = 442368
    kMDItemSecurityMethod = "None"
    kMDItemTitle = "Cognitive control in context: Working memory capacity and proactive control"
    kMDItemUseCount = 3
    kMDItemUsedDates = (
    "2021-07-13 04:00:00 +0000"
    )
    kMDItemVersion = "1.7"
  • adamsmith
    ("this is not our bug" is not defensive; it's just pointing users in the right direction)
  • dstillman
    @olivierroymd: As adamsmith says, explaining that something isn't a bug in Zotero isn't being defensive — it's helping you understand whom you would have to contact to actually get it fixed. We can't fix something that isn't our bug.
    A quick Google search seems to relate this issue to individual apps responsible of "putting the file there" in the file system. Therefore, it would seem, it looks like a Zotero issue and the ability to set this metadata attribute should maybe be built in?
    I don't know what guidance you're referring to — you'd have to provide a link — but no, this is an attribute managed by the system, not something every app is supposed to set. From Apple's own documentation:
    kMDItemLastUsedDate: The date and time that the file was last used. This value is updated automatically by LaunchServices everytime a file is opened by double clicking, or by asking LaunchServices to open a file.
    Testing on Big Sur, neither Chrome nor Firefox sets this when saving a PDF. It somehow gets set when using File → Save As… in Safari but not when using the download button in Safari's internal PDF viewer. Similarly, it gets set when dragging a file to a folder in Transmit — presumably because of the interaction with the Finder — but not when simply using right-click → Download. It obviously doesn't get set when using command-line tools like wget or curl. There are countless programs with all kinds of underlying technologies that can create files on disk, and it would be absurd if they were all supposed to set an extended metadata attribute on the file for the OS's search system to work correctly. (I'm not sure we even could set it if we wanted to, which we definitely don't.)

    Again, this isn't our bug, and if you want it fixed, you should go talk to the Alfred developers or Apple and ask them to support finding files that haven't yet been opened.
  • olivierroymd
    Interestingly, despite the Zotero folks saying that Zotero has nothing to do with my problem, I now can get to every pdf using Alfred after moving from the Zotero Sync service to a simple folder on my drive without the sync functionality.
  • dstillman
    I explained the technical details above. As I say, and as Apple's documentation shows, it's not Zotero's job to set this attribute. I have no idea why you're arguing with us about this.
  • olivierroymd
    Once again, I don't get the defensive tone. All I am saying is this: can something be done about the fact that, when Zotero full text sync is on (which I am paying for storage, BTW), my workflow is interrupted by the fact that I cannot reach PDFs files with Alfred? In other words, how can Zotero sync not interfere with this process when, if I don't use the full text sync feature, Alfred can find all the PDFs with ease?

    To make things clear, Alfred developers also say this is not their bug.

    All I want is to be able to use the full text sync feature and be able to reach PDFs with Alfred.
  • dstillman
    Do you genuinely not understand that our explaining that something isn't our bug is not being defensive? We literally have no way to help you here, and we are explicitly telling you whom you would need to talk to fix the bug, which — one more time — is not a bug in Zotero.

    I have no idea how Alfred search works, but we obviously have no control over it. If they use their own search index, only they can help you. If they use Spotlight under the hood, then it's an Apple bug, as explained above, and you or they would need to contact Apple. I gave you a slew of examples in which apps, including Apple's own apps, don't set this attribute, and I linked to documentation from Apple explaining that it's the system's job to manage it as files are opened. I honestly have no idea what more you want from us, but we can't help you further, and you'll need to talk to the people responsible for the bug.
Sign In or Register to comment.