Memory leak of closed PDF Items with 100 more annotations when plugin enabled

trarer
edited August 22, 2023
Once the extension(already tried many different plugins) is enabled in zotero 7 for macOS, the memory of closed PDF pages with 100 or more annotations will not be released and will continue to accumulate. Continuously open more than a dozen different PDFs with 100 more annotations, keep annotations sidebar open and then close them, and the memory quickly reaches 5 to 6 GiB.The memory report shows that the bulk of the memory is occupied by the process named explicit/gfx/heap-textures.
  • dstillman
    edited August 21, 2023
    What do you mean by "with extension enabled"? Are you talking about plugins? A plugin that doesn't have anything to do with the reader shouldn't have any effect, and a memory leak in a plugin is a problem for the plugin developer to fix.
  • trarer
    But the memory leak problem only exists on the macOS, and this kind of memory leak caused by plugins is very common. I don't know if this is a memory management flaw in macOS or a code bug in zotero.
  • dstillman
    I still don't know what you're talking about.
  • trarer
    If the plugin itself causes a memory leak, it will exist on both windows and macOS. But currently only macOS M2 has this problem. I don't know where to go for help, now I either disable these plugins or use zotero for a while and restart it.
  • dstillman
    edited August 21, 2023
    Please take a step back and explain the actual problem you're facing, with specific details. What plugins are you even talking about?

    See Steps to Reproduce for the kind of description we need to do anything with this.
  • trarer
    windingwind/zotero-pdf-translate, l0o0/jasminum, MuiseDestiny/zotero-style,
    MuiseDestiny/zotero-reference and so on. Each of the above plugins will cause a memory leak.
  • dstillman
    edited August 21, 2023
    Please provide steps to reproduce, with a single plugin enabled.
  • trarer
    edited August 21, 2023
    Report ID:475751934

    When zotero 7 on macOS M2 is started with a plugin enabled, the memory of closed PDF pages will not be released and will continue to accumulate. Continuously open more than a dozen PDFs and then close them, and the memory quickly reaches 5 to 6 GiB.The memory report shows that the bulk of the memory is occupied by the process named explicit/gfx/heap-textures.

    Steps to reproduce:

    1. Start Zotero.
    2. Click "Tools/Add-ons"，enable a plugin such as "windingwind/zotero-pdf-translate".
    3. In My Library, open a PDF item, scroll it from beginning to end and close.
    4. Open another PDF item, scroll it from beginning to end and close.
    5. Repeat 4 about 10 times.
    6. Click "Tools/Developer/Run javaScript"，run "Zotero.openInViewer('about:memory')".
    7. In "Show memory reports", click "measure".

    Repoort show:

    Main Process (pid 10142)
    Explicit Allocations

    3,418.67 MB (100.0%) ++ explicit

    Other Measurements

    19.58 MB (100.0%) ++ decommitted

    8,009 (100.0%) ++ event-counts

    1 (100.0%) -- extensions
    └──1 (100.0%) ── Extension(id=zoteropdftranslate@euclpts.com, name="Translate for Zotero", baseURL=moz-extension://6bfbfaed-2707-4234-a220-8dd898ed6dd6/)

    1 (100.0%) -- file-blob-urls
    └──1 (100.0%) ── owner unknown/blob:null/88c161c0-5da6-4e80-b3d7-b59a0a2efb2e

    1,209.14 MB (100.0%) ++ gfx

    18 (100.0%) ++ ghost-windows

    3,246.20 MB (100.0%) ++ heap-committed

    8.01 MB (100.0%) ++ images

    3 (100.0%) ++ ipc-channels

    3 (100.0%) ++ ipc-channels-peak

    122 (100.0%) ++ js-component-loader

    8 (100.0%) ++ js-helper-threads

    602.43 MB (100.0%) ++ js-main-runtime

    143.97 MB (100.0%) ++ js-main-runtime-gc-heap-committed

    152 (100.0%) ++ js-main-runtime-realms

    20 (100.0%) ++ message-manager

    1,326 (100.0%) ++ observer-service

    820 (100.0%) ++ observer-service-suspect

    204 (100.0%) ++ preference-service

    0.14 MB (100.0%) ++ shared-string-bundles

    58.76 MB (100.0%) ++ window-objects

    0.25 MB ── font-list-shmem
    0.00 MB ── gfx-textures
    0.00 MB ── gfx-textures-peak
    0.00 MB ── gfx-tiles-waste
    18 ── ghost-windows
    3,151.89 MB ── heap-allocated
    1.00 MB ── heap-chunksize
    3,473.00 MB ── heap-mapped
    0 ── imagelib-surface-cache-already-present-count
    2.60 MB ── imagelib-surface-cache-estimated-locked
    2.63 MB ── imagelib-surface-cache-estimated-total
    256 ── imagelib-surface-cache-image-count
    239 ── imagelib-surface-cache-image-surface-count
    254 ── imagelib-surface-cache-locked-image-count
    237 ── imagelib-surface-cache-locked-surfaces-count
    0 ── imagelib-surface-cache-overflow-count
    0 ── imagelib-surface-cache-table-failure-count
    2 ── imagelib-surface-cache-tracked-cost-count
    2 ── imagelib-surface-cache-tracked-expiry-count
    0 ── imagelib-surface-cache-tracking-failure-count
    7.72 MB ── js-main-runtime-temporary-peak
    393 ── page-faults-hard
    3,498,800 ── page-faults-soft
    752.11 MB ── resident
    2,289.75 MB ── resident-peak
    187.20 MB ── resident-unique
    0.00 MB ── shmem-allocated
    17.67 MB ── shmem-mapped
    0 ── unresolved-ipc-responses
    425,847.00 MB ── vsize
    0.00 MB ── wasm-runtime
  • hsiangyu_wong
    What is your plugin version and Zotero version? I cannot reproduce what you claimed with your steps.

    After open-close a pdf (4.4MB) 10 times, the memory usage goes back to 600-700 MB, and about:memory shows the main process's explicit allocations are 347MB.

    Test on Ventura 13.5 (M2 Pro), Z7 beta36, pdf-translate 1.1.0-10 (prerelease).
  • trarer
    Maybe I didn't describe the steps clearly. Repeatedly open-scroll-close the same PDF file may not be reproduced, and you may try multiple different PDFs.
  • hsiangyu_wong
    That does not make any difference to me. After open-scroll-close 10 different PDFs, about:memory shows 460MB of the main process and the total memory usage is 850MB. I test in a much larger library this time so a little increment of memory usage is reasonable.
  • trarer
    It is tested on macOS Ventura 3.5.1 (M2), Zotero 7 beta 36, pdf-translate 1.1.0-10 (prerelease). It can't be a problem with the database or the file itself, right? My Zotero 7 is installed over the old Zotero 6 version.
  • hsiangyu_wong
    I can't help right now, but let's see if others have the same problem.
  • trarer
    edited August 21, 2023
    Thanks for help. The problem seems to be resolved for now. Neither throwing the software in the trash and reinstalling nor deleting the database seemed to work. I tried to use software such as App Cleaner to completely uninstall zotero and its related residual files, and re-login with the account, it seems that the problem has been solved. The specific reason for the memory leak is unknown, but if zotero 7 is installed on zotero 6, some strange problems will indeed appear.
  • trarer
    edited August 21, 2023
    Report ID: 1742294236

    When I did a clean uninstall and install, the memory leak problem seemed to be resolved. But after using it for a while, the same problem appeared again. But this time I found the reason why the problem seems to be solved. When viewing a PDF, if the sidebar is opened, the problem recurs. The reproduction steps are modified as follows.

    Steps to reproduce:

    1. Start Zotero.
    2. Click "Tools/Add-ons"，enable a plugin such as "windingwind/zotero-pdf-translate" and restart zotero.
    3. In My Library, open a PDF item, toggle sidebar to make it open, scroll pdf from beginning to end and close.
    4. Repeat 3 about 10 more times for different pdf items.
    5. Click "Tools/Developer/Run javaScript"，run "Zotero.openInViewer('about:memory')".
    6. In "Show memory reports", click "measure".

    It shows:

    Main Process (pid 1333)
    Explicit Allocations

    3,083.33 MB (100.0%) -- explicit
    ├──2,184.29 MB (70.84%) -- gfx
    │ ├──2,180.53 MB (70.72%) ── heap-textures
    │ └──────3.76 MB (00.12%) ++ (5 tiny)
    ├────508.23 MB (16.48%) -- window-objects
    │ ├──444.93 MB (14.43%) -- top(none)
    │ │ ├──267.11 MB (08.66%) -- ghost
    │ │ │ ├──231.91 MB (07.52%) -- window(resource://zotero/reader/pdf/web/viewer.html)
    │ │ │ │ ├──220.38 MB (07.15%) -- js-realm(resource://zotero/reader/pdf/web/viewer.html, about:blank)
    │ │ │ │ │ ├──215.97 MB (07.00%) -- classes
    │ │ │ │ │ │ ├──151.89 MB (04.93%) -- class(ArrayBuffer)/objects
    │ │ │ │ │ │ │ ├──151.32 MB (04.91%) ── malloc-heap/elements/normal [8]
    │ │ │ │ │ │ │ └────0.57 MB (00.02%) ── gc-heap [8]
    │ │ │ │ │ │ └───64.08 MB (02.08%) ++ (11 tiny)
    │ │ │ │ │ └────4.41 MB (00.14%) ++ (2 tiny)
    │ │ │ │ └───11.53 MB (00.37%) ++ (3 tiny)
    │ │ │ └───35.21 MB (01.14%) ++ window(resource://zotero/reader/reader.html)
    │ │ └──177.82 MB (05.77%) -- detached
    │ │ ├──157.97 MB (05.12%) -- window(resource://zotero/reader/pdf/web/viewer.html)
    │ │ │ ├──153.03 MB (04.96%) -- js-realm(resource://zotero/reader/pdf/web/viewer.html, about:blank)
    │ │ │ │ ├──150.81 MB (04.89%) -- classes
    │ │ │ │ │ ├──114.92 MB (03.73%) -- class(ArrayBuffer)/objects
    │ │ │ │ │ │ ├──114.82 MB (03.72%) ── malloc-heap/elements/normal [4]
    │ │ │ │ │ │ └────0.10 MB (00.00%) ── gc-heap [4]
    │ │ │ │ │ └───35.89 MB (01.16%) ++ (11 tiny)
    │ │ │ │ └────2.22 MB (00.07%) ++ (2 tiny)
    │ │ │ └────4.94 MB (00.16%) ++ (3 tiny)
    │ │ └───19.85 MB (00.64%) ++ window(resource://zotero/reader/reader.html)
    │ ├───31.83 MB (01.03%) ++ top(chrome://zotero/content/zoteroPane.xhtml, id=3)/active/window(chrome://zotero/content/zoteroPane.xhtml)
    │ └───31.47 MB (01.02%) ++ (7 tiny)
    ├────147.26 MB (04.78%) -- js-non-window
    │ ├───98.57 MB (03.20%) -- zones
    │ │ ├──56.34 MB (01.83%) ++ zone(0x10e652000)
    │ │ └──42.23 MB (01.37%) ++ (7 tiny)
    │ ├───43.79 MB (01.42%) ++ runtime
    │ └────4.90 MB (00.16%) ++ (4 tiny)
    ├─────98.47 MB (03.19%) ── heap-unclassified
    ├─────81.06 MB (02.63%) ++ (28 tiny)
    └─────64.03 MB (02.08%) -- heap-overhead
    ├──38.46 MB (01.25%) ++ bin-unused
    └──25.57 MB (00.83%) ++ (2 tiny)


    I don't know if everyone can reproduce this time？
  • poettli
    edited August 22, 2023
    Zotero 7.0.0-beta.36
    Apple M2 Ventura 13.5.1 (22G90)
    windingwind/zotero-pdf-translate 1.1.0-11 activated

    I (correctly?) have performed step #3 a dozen of times with different PDFs (OCRed included). I can see the memory being released after I close the PDF.
  • trarer
    Thanks for testing. Are you sure you have opened the sidebar with PDF annotations? I tried again, trying to find the cause. I feel that there may be a memory leak caused by the PDF annotation in the sidebar. Opening a PDF with a lot of annotations can cause this problem. But I'm not sure, because the previous tests were to open PDFs with more annotations, and it seems that the problem is not so serious when opening PDFs without annotations.
  • hsiangyu_wong
    Again, no difference for me. After closing all pdf tabs and waiting for ~1 minute, the memory usage goes down to normal. the annotation left sidebar is kept open.

    GC does not happen instantly. Please wait for a while and see if the memory usage is still high.
  • trarer
    edited August 22, 2023
    The problem still exists. I waited for more than 10 minutes, but the memory usage is still very high. 2,279.67 MB heap-textures, 3,169.64 MB heap-committed. Manually clicking GC or CC doesn't work either.
  • trarer
    Report ID: 1742294236

    After repeated testing for a period of time, I found that the memory leak problem may be caused by a large number of PDF annotations and sidebar annotations. The reproduction steps are modified as follows.

    Steps to reproduce:

    1. Start Zotero.
    2. Click "Tools/Add-ons"，enable a plugin such as "windingwind/zotero-pdf-translate" and restart zotero.
    3. In My Library, make sure there are a dozen different PDF items with hundreds of annotations.
    4. open a PDF item with 100 more annotations, toggle annotation sidebar to make it open, scroll pdf from beginning to end and close.
    5. Repeat 4 about 10 more times for different PDF items.
    6. Click "Tools/Developer/Run javaScript"，run "Zotero.openInViewer('about:memory')".
    7. In "Show memory reports", click "measure".

    Hoping to see if someone can reproduce it and help me find the cause.

    Tested on macOS Ventura 3.5.1 (M2), Zotero 7 beta 36, pdf-translate 1.1.0-11 (prerelease)
  • hsiangyu_wong
    Is it possible the issue with the annotation sidebar isn't related to plugins? Have you tried replicating the problem with no plugins enabled?
  • trarer
    Of course, I have tried not to install any plugins, and the software is fine at this time. I guess many plugins may have some connection with PDF annotation. I don't know if you have reproduced this problem, although it's a little troublesome.
  • yokberg
    the same problem on macos13.2,zotero7.0.0-beta.38, I use zotero-night plugin, an I also encountered a memory leak issue. But if I disable this plugin, memory will be properly released.

    so I think the problem is with the plugins.The plugins don't work well with zotero.
  • hsiangyu_wong
    OK, I will check. Thanks for the report.
  • Raphazoni
    I've been having a memory issue on my Zotero.
    Closing and opening the app ameliorates.
    I have LibreOffice and Word-Mac OS integration extensions. Are those plug-ins, the ones you've been talking about?
  • trarer
    The phenomenon of macOS zotero not releasing memory after closing PDF seems to only exist on hosts with small memory, such as 8GiB. I guess this may be related to the memory compression technology of macOS. From the Activity Monitor, you can see that after closing all PDFs, the memory occupied by zotero is mainly concentrated in the compressed memory. Of course this is just speculation.
  • trarer
    The same memory leak problem occurs on Linux Zotero.
Sign In or Register to comment.