If you use a web mail client you may be able to use the "Create New Item From Current Page" button, or highlight text and chose "Create Zotero Note and Item From Selection". Otherwise copy paste is probably your best bet.
Doesn't "Create New Item From Current Page" also save all the graphics and even the JavaScript to display the ads on that same page? Wouldn't that bloat your database quite a bit, just to save a few paragraphs of text?
you have ads on your webpages? That's what FF/adblock is for...
But generally you're right, but Zotero assumes (imho correctly) that harddisk space has practically become a non-issue - that's why it's taking snapshots of everything. Even a huge database with 100.000 entries and a wildly exaggerated 100k per file produces a mere 10Gb of data - very little considering today's hd size.
Note that Zotero, because it just "deposits" the files is not going to slow down bc it's database is slower in terms of kb - relevant is only the number of entries.
Yes, I use the NoScript add-on for Firefox to block ads. However, for these issues I think in terms of how things would work for regular people. As a former computer consultant, I can tell you that most people have no clue how to get rid of all those ads.
I am not so much concerned with hard drive space as I am with database bloat. The larger the database, the more there is to index and search through later. The larger the database, the longer it takes to sync (assuming eventual upgrade to 1.5 or later). The larger the database, the longer it takes to do just about anything. Even if Zotero and the underlying database are very efficient, there is no sense in being exorbitantly wasteful. Besides, what if someone wanted to use Zotero on one of those new NetBooks that are coming out. They don't have gigantic hard drives. Or, what if someone wanted to store their Zotero database on a thumb-drive. We may not have to write code in Assembler just to squeeze out every last CPU cycle, but there is no sense in being so wasteful that we collect and store 100 times more data than we actually need.
Of course, I realize how difficult it would be to write code that filters out all of the advertising before placing the HTML in the database. It ends up being a never-ending battle with the ad placement firms and it would bloat the Zotero code. NoScript works for me but only because I manually sort out which JavaScript to allow and which to forbid. Most users would not be able to do that. So, I do not expect the Zotero programmers to take on the task of figuring out what is and is not extraneous fluff on a web page. I only made my original point so that users could decide for themselves whether they want to store all that extra stuff on that web page or simply the text of the e-mail. In my view: An informed user is a happy user.
Besides, I have a suspicion that the original poster was looking for a way for Zotero to automatically collect all the necessary information to create a citation from the e-mail and fill it into the fields. As it is, if a user wants to use an e-mail in an academic paper, then they have to manually fill in all the fields in the Zotero item in the database. This can become tedious (even with cutting and pasting) if the researcher is gathering a lot of material through direct e-mail conversations.
Again, I realize that this would add a slew of additional translations that Zotero would have to do in order to be able to scrape the data off of all the different types of web-based e-mail interfaces. In addition, it would be much more difficult keeping up with all the changes that web-based e-mail service providers make to their web sites than the relatively few changes made to academic search sites.
Be that as it may, I am sure that many people would find that to be a very handy feature.
OK with evertything you say, but the automatic adblock you describe already exists as a FF add-on:
http://adblockplus.org/en/
especially if used with a filter subscription it requires zero maintenance by the user beyond installing.
I just tried this out with (and add free version of) gmail and I'll admit the files are much bigger than I imagined (i.e. the files sum up to more than 1MB), so you're probably right this isn't a terribly good idea
the result isn't all that bad, though, here is the citation it produces.
@u.northwestern.edu Mail - The NBER Digest -- March 2009 - Available at: https://mail.google.com/a/u.northwestern.edu/[...] [Accessed February 27, 2009].
edit: the e-mail import may be ideal to write as a plugin, no?
edit: the e-mail import may be ideal to write as a plugin, no?
I'd love to, just as soon as I really learn how to write code. I am still at the learning data structures and stuff like Java Generics right now. But I've got less than a year to really get up to speed if I want to achieve some of my other goals.
Anyway, so does that AdBlockPlus add-on really prevent the extra files from even being downloaded? Does it prevent the JavaScript that downloads the ads from being stored in the Zotero storage folder? Or does it just prevent the ads from being displayed both when you first view the page and later when you view the Zotero snapshot? If the latter, then if you later export that database and give it to someone who doesn't have AdBlockPlus would they then see the ads? Its not the biggest concern but it is something to think about.
So, I tried the trick of simply clicking "Show File" and then editing that HTML page. I was able to delete all the extraneous stuff from the web page itself but was unable to easily get rid of the unused files. Dreamweaver does have a feature to delete unused files but it only works for complete "sites" that you have created in the program. It does not work for a single, stand-alone web page. It would be too much trouble to create a site in Dreamweaver, edit the page, use the Site Checker to find and delete the unused files, then delete the "site" from Dreamweaver's list of "sites" that it knows about. It is also too much trouble to attempt to delete each file in turn and see if Dreamweaver complains. So I won't be using Dreamweaver to clean up the web pages stored in the Zotero storage folders.
Your idea is solid. Now I just need some kind of HTML editor that will allow me to:
Open the file just by double clicking on the file in the Zotero folder.
Select the part of the web page I want to keep.
Invert that selection.
Then delete all the part I don't want to keep.
This should then also delete all the files that were linked to by the part deleted (unless they are used by some part that wasn't deleted, of course).
This would be the ideal solution but another option would be to just find an HTML editor that would let me find and delete files not used within just that one specific web page instead of only doing that for entire "sites."
Anyway, so does that AdBlockPlus add-on really prevent the extra files from even being downloaded?
Yes. Zotero only sees the page as it is presented after AdBlockPlus has blocked the ads. So the ads won't be saved to your hard drive.
As for your other points, it might be worthwhile to check out some Firefox extensions. The one below allows you to do the selection inversions. It also has the benefit that you can strip the page before saving it in Zotero, saving you the trouble of firing up an external HTML editor each time.
As for your other points, it might be worthwhile to check out some Firefox extensions. The one below allows you to do the selection inversions. It also has the benefit that you can strip the page before saving it in Zotero, saving you the trouble of firing up an external HTML editor each time.
I checked out this add-on and it is truly wonderful. It does make all of the stuff I don't want invisible. However...
Zotero only sees the page as it is presented after AdBlockPlus has blocked the ads. So the ads won't be saved to your hard drive.
Doesn't seem to be exactly the case. Go to any web page with text and pictures. Select only one paragraph of text. Now right-click on it and choose "Remove Everything Else." Now make a Zotero item out of it by clicking the "Create New Item from Current Page" button in Zotero. Now expand the item, select the snapshot, then click the "Show File" button in Zotero. You will see that Zotero still collected lots of additional files. It gathers up all the graphics and javascript files whether they were part of what was left displayed on the page after using Nuke Anything Enhanced or not.
I appreciate your attempts to solve this issue. However, it seems that without a major change to WebPageDump, this behavior won't be changing any time soon. It should still be possible for me to find some kind of simple HTML editor that will do what I want quickly, without any changes to Zotero or WebPageDump. When I find something, I will let you know.
My mistake. I assumed (wrongly) that Zotero wouldn't slip behind AdBlock's back and download the original page if you click "Create new item from current page".
My mistake. I assumed (wrongly) that Zotero wouldn't slip behind AdBlock's back and download the original page if you click "Create new item from current page".
My gues is that it is the WebPageDump code. It is probably simply looking at the parent URL and grabbing everything referenced in the original HTML file. It is odd though, that it still uses the edited HTML from AdBlock or Nuke Anything Enhanced. Oh well, it will sort itself out eventually.
WebPageDump traverses the actual state of the DOM tree not(!) the original URL (in its base configuration it skips JavaScript code but I think this was changed for zotero).
Beside this the "Scrapbook" extension allows saving web page snippets - in fact WebPageDump is originally based on the scrapbbook saving code - http://amb.vis.ne.jp/mozilla/scrapbook/
"
bernhard wrote:
"Beside this the "Scrapbook" extension allows saving web page snippets - in fact WebPageDump is originally based on the scrapbbook saving code -"
You can use Scrapbook's "Edit before capture" feature on a website, get rid of anything you don't want with the DOM eraser, then switch to Zotero and take a snapshot of the cleaned website.
The items you erased will not be present in the Zotero snapshot files afterwards (as far as I can see).
Upgrade Storage
But generally you're right, but Zotero assumes (imho correctly) that harddisk space has practically become a non-issue - that's why it's taking snapshots of everything. Even a huge database with 100.000 entries and a wildly exaggerated 100k per file produces a mere 10Gb of data - very little considering today's hd size.
Note that Zotero, because it just "deposits" the files is not going to slow down bc it's database is slower in terms of kb - relevant is only the number of entries.
I am not so much concerned with hard drive space as I am with database bloat. The larger the database, the more there is to index and search through later. The larger the database, the longer it takes to sync (assuming eventual upgrade to 1.5 or later). The larger the database, the longer it takes to do just about anything. Even if Zotero and the underlying database are very efficient, there is no sense in being exorbitantly wasteful. Besides, what if someone wanted to use Zotero on one of those new NetBooks that are coming out. They don't have gigantic hard drives. Or, what if someone wanted to store their Zotero database on a thumb-drive. We may not have to write code in Assembler just to squeeze out every last CPU cycle, but there is no sense in being so wasteful that we collect and store 100 times more data than we actually need.
Of course, I realize how difficult it would be to write code that filters out all of the advertising before placing the HTML in the database. It ends up being a never-ending battle with the ad placement firms and it would bloat the Zotero code. NoScript works for me but only because I manually sort out which JavaScript to allow and which to forbid. Most users would not be able to do that. So, I do not expect the Zotero programmers to take on the task of figuring out what is and is not extraneous fluff on a web page. I only made my original point so that users could decide for themselves whether they want to store all that extra stuff on that web page or simply the text of the e-mail. In my view: An informed user is a happy user.
Besides, I have a suspicion that the original poster was looking for a way for Zotero to automatically collect all the necessary information to create a citation from the e-mail and fill it into the fields. As it is, if a user wants to use an e-mail in an academic paper, then they have to manually fill in all the fields in the Zotero item in the database. This can become tedious (even with cutting and pasting) if the researcher is gathering a lot of material through direct e-mail conversations.
Again, I realize that this would add a slew of additional translations that Zotero would have to do in order to be able to scrape the data off of all the different types of web-based e-mail interfaces. In addition, it would be much more difficult keeping up with all the changes that web-based e-mail service providers make to their web sites than the relatively few changes made to academic search sites.
Be that as it may, I am sure that many people would find that to be a very handy feature.
http://adblockplus.org/en/
especially if used with a filter subscription it requires zero maintenance by the user beyond installing.
I just tried this out with (and add free version of) gmail and I'll admit the files are much bigger than I imagined (i.e. the files sum up to more than 1MB), so you're probably right this isn't a terribly good idea
the result isn't all that bad, though, here is the citation it produces.
@u.northwestern.edu Mail - The NBER Digest -- March 2009 - Available at: https://mail.google.com/a/u.northwestern.edu/[...] [Accessed February 27, 2009].
edit: the e-mail import may be ideal to write as a plugin, no?
Anyway, so does that AdBlockPlus add-on really prevent the extra files from even being downloaded? Does it prevent the JavaScript that downloads the ads from being stored in the Zotero storage folder? Or does it just prevent the ads from being displayed both when you first view the page and later when you view the Zotero snapshot? If the latter, then if you later export that database and give it to someone who doesn't have AdBlockPlus would they then see the ads? Its not the biggest concern but it is something to think about.
So, I tried the trick of simply clicking "Show File" and then editing that HTML page. I was able to delete all the extraneous stuff from the web page itself but was unable to easily get rid of the unused files. Dreamweaver does have a feature to delete unused files but it only works for complete "sites" that you have created in the program. It does not work for a single, stand-alone web page. It would be too much trouble to create a site in Dreamweaver, edit the page, use the Site Checker to find and delete the unused files, then delete the "site" from Dreamweaver's list of "sites" that it knows about. It is also too much trouble to attempt to delete each file in turn and see if Dreamweaver complains. So I won't be using Dreamweaver to clean up the web pages stored in the Zotero storage folders.
Your idea is solid. Now I just need some kind of HTML editor that will allow me to:
- Open the file just by double clicking on the file in the Zotero folder.
- Select the part of the web page I want to keep.
- Invert that selection.
- Then delete all the part I don't want to keep.
- This should then also delete all the files that were linked to by the part deleted (unless they are used by some part that wasn't deleted, of course).
This would be the ideal solution but another option would be to just find an HTML editor that would let me find and delete files not used within just that one specific web page instead of only doing that for entire "sites."Does anyone know of such a program?
As for your other points, it might be worthwhile to check out some Firefox extensions. The one below allows you to do the selection inversions. It also has the benefit that you can strip the page before saving it in Zotero, saving you the trouble of firing up an external HTML editor each time.
Nuke Anything Enhanced
https://addons.mozilla.org/en-US/firefox/addon/951
I appreciate your attempts to solve this issue. However, it seems that without a major change to WebPageDump, this behavior won't be changing any time soon. It should still be possible for me to find some kind of simple HTML editor that will do what I want quickly, without any changes to Zotero or WebPageDump. When I find something, I will let you know.
Beside this the "Scrapbook" extension allows saving web page snippets - in fact WebPageDump is originally based on the scrapbbook saving code - http://amb.vis.ne.jp/mozilla/scrapbook/
bernhard wrote:
"Beside this the "Scrapbook" extension allows saving web page snippets - in fact WebPageDump is originally based on the scrapbbook saving code -"
You can use Scrapbook's "Edit before capture" feature on a website, get rid of anything you don't want with the DOM eraser, then switch to Zotero and take a snapshot of the cleaned website.
The items you erased will not be present in the Zotero snapshot files afterwards (as far as I can see).