Upgrade StorageAny test cases available?
Is there anyone out there willing to share a set of test cases that they use for testing their CSL?
It would be great if we could have an exported "Zotero RDF" or zotero.sqlite with test documents (various combinations of empty/non-empty fields across all document types).
It would be great if we could have an exported "Zotero RDF" or zotero.sqlite with test documents (various combinations of empty/non-empty fields across all document types).
I just want a couple of references that let me see quickly if I've done everything right in a style I'm working on. This appears to be something else, but maybe I'm also just not getting it?
To test a change made to a style using the test cites in mark's group, you would (I think) load the modified style to Zotero, then select a suitable test entry in Zotero, then call up the csledit.xul page, and then select the style you're testing, then inspect the output and judge whether it does what you expected. That tests one instance of behavior. To test other behavior, you would select other entries, run the style, and again inspect the results visually, and compare them with your expectations. This method of testing leaves behind no lasting record, and can't be repeated without going through the same point-and-click steps all over again.
Using something like the processor tests (i.e. a program that you run from the command line that processes the test data outside of Zotero), the task of checking the output for correctness can be automated, and the tests can be batched together into a single "style test". Each time you make a change to the style, you would also write one or more tests to make sure that everything is working, and add them to the style's test bundle. Then you would run the entire bundle of tests, to be sure that they all return "ok". That lets you lock in the current behavior of the style, and protects against breakage that might be caused by later revisions to the style CSL or the processor.
Having a good, full set of tests for each of the major styles is an essential step toward the widespread adoption of CSL as a standard tool that publishers positively recommend to their editors and authors. I would be very happy to help build a test framework and bundling architecture that is tailored to the needs of style testing.
What I do is to go into the test panel, select all of the items, and see if they do what I want. As long as I'm working in the chrome panel - which I always use for small edits - the updating is automatic and I have all of the result in the lower part of my screen - so it's actually a lot less complicated than you indicate.
For style creation purposes I'm not sure how that can be more automated, because obviously the program doesn't know (yet) what I want from the style.
What you seem to talk about is automatic checking for existing styles (i.e. all major styles as you say) - that's out of my league.
1/ All test citation documents packaged/bundled in a single file for an easy import into zotero.
2/ A holistic approach to building the set of test cases toward covering all possible combinations of item types and field contents.
Bruce, sorry for the ambiguity. I am not a CSL-processor developer. I am interested in modifying CSL styles so I am looking for test cases to test the CSL file content.
Adam, I think we are on the same page.
Frank, thanks for the elaboration. I you have any capacity to build such a framework, it would definitely be appreciated. For my purposes (regression test of a specific CSL file) at the moment I hoped to use the chrome://zotero/content/tools/csledit.xul page with all the items in the test citation library selected.
2) OK, that would require a more systematic approach - suggestions on how to proceed? All combinations doesn't make sense - that's too much stuff to look at.
1) excuse my ignorance, I have not used the group feature yet.
2) I agree, all combinations would be an overkill. On the other hand it is also difficult to determine which ones are possible/make sense. I hope someone can come up with good suggestions. Maybe we could review available style guides and compile a superset of examples and then try to extend them.
Journal Article
Book
Book Section
Webpage
Presentation
Conference Paper and
Document
I think for most of these there are common features to be left out - e.g. a Journal Article will always have a Journal title, and a title, the most crucial things that can miss are issue, volume, and page numbers.
There is significant overlap for some parts - we only need to test what happens without author and without year for one item - regardless of type.
These are the four that I've already put in the group (in APSA style). The respective title tells you what you're supposed to see.
Author1, One. Book With Series Title, 3rd Edition, no date. Testville: Test Publisher.
Author1, One, Two Author2, and Three Author3. “Article, 3 Authors, abbreviation given..” Test Journal 12(2): 12-17.
Author1, One, Two Author2, and Three Author3. 2008. “Article, no issue, no pages, URL, full date..” Test Journal 12. Available at: www.testurl.edu [Accessed December 16, 2008].
Author1, Otherone, and Two Author2. 2001. “Book Section, 2 Authors, 2 Editors, 1st. Volume.” In Edited Volume Book, eds. One Editor1 and Two Editor2. Testplace: Test Publisher, p. 1-17.