"Automatically rename [...]" does indeed rename but doesn’t show it (beta)
When added through the Safari Zotero Connector in beta, PDFs are renamed but the name of the PDF isn’t updated in Zotero’s interface (e.g. I still see "Full Text PDF" as the PDF file). Browsing the folder where the file is located confirms that the PDF has indeed been renamed. Closing/reopening Zotero or syncing had no effect.