How to import data

Yuzuke

Hi,

for a little programming project i need to know how Zotero is import data and structureing it. My Professor said i had to use the Zotero database sheme for the project since it is a really good one.

What i have to do is write scripts do import data about scientific papers into my own database. So basically i copied the sheme of Zotero and set up my own database on mysql. Now i am thinking of how to tell the Script / Parser how to import the data. For example: How does Zotero know to put the String "Year 2012" into the column "published-year"? I searched over Zotero-Code on Github but i haven't found anything yet.

Can you please tell me where i can find the way Zotero is doing this so i am able to use this code as a little tip for my own programming project?

Thank you!

mronkko

Your question is a bit difficult to understand. If you could provide more context, that would allow people to provide a better response. Modifying the database directly is not necessarily the best option.

You should probably check this project to see if there is any code that you can reuse.

http://www.cogsci.nl/software/qnotero
https://github.com/smathot/qnotero

Yuzuke

Hi there and thanks for your replay,

What i mean is that i am trying to build an own import script to take Data from scientific Databases and put them into my own database. One example: The database pubmed is an open database which got basic bibliographic data about thousands of articles from medical background. So if i tell my script to import the data of 1000 medical articles from pubmed into my own database how does that work? How does my script know which pubmed data needs to be stored exactly where in my own database?

This: http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&id=1

does export the basic bibliographic data of the article with id one in XML format to my php script. So i got all that stored in my php script in the variable $xml. But how do i tell my script to "put the value of the published year" into my database to the corresponding field? How does the script find matches? I tried to pull it off my own and started to write an own php parser but its really hard to get it working. For that i wanted to see how Zotero is doing it and maybe i can resuse some of zoteros code.

Thanks for your links i will check them now.

mronkko2

Check this page for how the translators work:

http://www.zotero.org/support/dev/translators/coding

Yuzuke

OK, i think i got it now. Next problem would be to set up the complete database as it is in Zotero. I found http://zomark.github.com/zotero-marc/schema/trunk/index.html which is the sheme Zotero uses right? Is there a finished MySQL Script which sets up this sheme in my local database? Like if i start the script it runs and and after it is finished i have all those tables and the complete sheme in my DB? It would save me a lot of trouble to manually set the database up as it is documented in the link. Thanks a lot!

mronkko2

You can use command line sqlite and dump the DB structure in a file. Then just run that file to initialize the other DB. This will give you a blank Zotero database.

Also, you can check the Zotero source in the github to see how it constructs the database. See these files in the Zotero source code.

/system.sql
/triggers.sql
/userdata.sql

Yuzuke

Hm, the problem is, that i can't use Zotero on my University's PC due to Admin rights / Proxy settings etc. I just need the database sheme to work with it. But i can't access the standalone client here to get to the sheme.

For the second possibility i searched for the system, triggers and userdata.sql and found them. Then i went to my phpmyadmin executing their sql code. system.sql worked, the other 2 not. I am getting a huge pile of bugs trying to execute them.

Is there anyone outside who is kind enough to post me the complete and valid SQL code which i just need to execute in my phpmyadmin to get a blank zotero database? If not so what else should i do to get it? As i said it's not possible for me to start Zotero standalone client on this PC and i searched google for a long time now - no one ever posted the SQL Code for this. What shall i do?

fbennett

The source code for Zotero is all online in GitHub. You can either copy bits from the repo or make your own fork. In the code layout of the 3.0 branch, the SQL files are at the top of the tree:

https://github.com/zotero/zotero/tree/3.0

The Zotero database is built in sqlite. You may have problems building a database with the SQL source files in another SQL engine (I don't know, but there may well be sqlite-specific stuff in there).

It's not quite clear what your task is. Has your professor set you a training task to build a bibliographic storage container based on PHP, or are you looking for the least-effort path to build a working system with particular characteristics? If the latter, what are the requirements?

Recasting the Zotero database management layer in a completely different language would probably take several years of effort. You probably don't want to go there.

pipeacosta

Hi, I have a similar question: How can we reuse a zotero translator and what else do we need to populate our own publications database?

aurimas

Please post technical development questions to zotero-dev@googlegroups.com You will need to provide a lot more detail about what exactly you are trying to do if you want someone to give you a proper answer.