No. I developed the fallback system in the original version, before we had the cs:choose structure. This makes it redundant, and so adds unnecessary complexity for implementers. We have thus removed that (BTW, undocumented) processing behavior.
@erazlogo - A condition type="my-type" in Zotero 2.0 CSL evaluates to true when a type is "my-type" OR the type's fallback-type (super-type/catch-all-type) is "my-type". The static mapping of types to fallback types has been published in the forums before. One more compexity is that a type here is not the Zotero item type directly but a CSL Type that has been staticly mapped from the Zotero item type.
Zotero Item Type >> CSL Type >> CSL Fallback Type
@bdarcus - If you remove the fallbacks, what do you plan to do about the existing styles that will thus get broken?
Also, will the mapping from Zotero Item Type to CSL Type stay as is? I do have a style that requires to distinguish between the Web page type and Blog post and I was hoping that would be possible one day.
@bdarcus - I would wish to think it will not be a practical problem, however, why risk it now that Zotero community has managed to establish such a great number of citation styles that it can be be reckoned with in this regard in comparisons to other leading reference managers?
Quick sampling through the available styles shows a heavy use of "book", "chapter" and "article" type conditions which may have been knowingly (or unknowingly for that matter) relying on the fallback feature.
The problem with the fallbacks may simply be the actual naming of the types. I do not remember the times before the cs:choose structure but what I would expect now is that CSL types and Fallback types should not have the same names. An "article-journal" falling back on "article" is fine but a "book" falling back on "book" creates confusion.
My suggestion is:
1/ In CSL 1.0 keep the type processing logic as is in Zotero 2.0
2/ In CSL 1.1 modify csl-types.rnc (this is just a mock-up, hopefully backward-compatible and the part with the native types could be a Zotero-specific extension):
div {
cs-types =
cs-fallback-types
| cs-standard-types
| cs-native-types
}
div {
cs-fallback-types =
"article"
| "book"
| "chapter"
}
It is probably possible to cover this with the upgrade XSLT stylesheet. If I understand things correctly, the only case where fallback behavior currently takes effect is in conditionals. Every fallback-type (e.g. "book") in type-conditionals (e.g. <if type="book"/>) could be replaced by the fallback-type and all types that fall back to that fallback-type (so the previous example would change to <if type="book film artwork report bill case hearing statute audioRecording videoRecording computerProgram" match="any"/>). I agree it isn't very pretty, so some manual cleanup might be desirable afterwards. But style behavior would stay the same.
@Rintze - I think the fallbacks do help keep style files less wordy. But if you need to do away with them, your automatic transformation with XSLT is a good idea. Will you also update your CSL 1.0 specs to remove the fallback types, e.g. "article"?
The other problem I mentioned is about exposing the native types. Do I understand correctly from your example above that all Zotero item types, e.g. Computer Program, will be available in CSL? That would be great.
I would wish to think it will not be a practical problem, however, why risk it now that Zotero community has managed to establish such a great number of citation styles that it can be be reckoned with in this regard in comparisons to other leading reference managers?
CSL is independent of Zotero, by design. So it's a question of balancing different priorities here. In this case, the idea was to remove a now-unnecessary feature to make implementation simpler. As it is, it's already going to be a challenge to ensure that different processors are compliant (and so produce the exact same output).
I expect Zotero won't switch in the new 1.0-compatible processor until the upgraded styles work correctly.
@Rintze - What about the "article" type redundancy in CSL 1.0? If you remove the fallback feature, the "article" type should become invalid.
@bdarcus - Even when considering the CSL independence of Zotero...
1/ The fallback feature simplifies the citation style files and makes them easier to develop.
2/ The confusion about the fallback feature may be greatly reduced by using distinct names for types VS fallback types and by making the specification explicit about it.
3/ If a processor implementer cannot comply with the standard, they should simply document the incompatibility. In this particular case, the fallback feature implementation should be straightforward and the Zotero source code is available to everyone, including Mendeley and Pandoc.
4/ The mapping from the native types to CSL types is, too, a kind of a fallback layer.
Considering Zotero...
5/ Zotero is not in the beta stage (cf. Mendeley) and a smooth upgrade path will be crucial.
6/ Inability to access the native types in CSL 0.8 limits the level of compliance with some style guides and so makes the Zotero adoption difficult. Why not include more Zotero native types in the list of the standard CSL types or provide a native-type extension?
I hope there is still some space for a discussion on this toward the CSL 1.0 implementation or CSL 1.1 specification. Maybe the case could be made stronger either way if other style developers could contribute with their opinions.
People don't typically believe me when I say this, but It's not generally necessary to use types to condition formatting.
I hope there is still some space for a discussion on this toward the CSL 1.0 implementation or CSL 1.1 specification. Maybe the case could be made stronger either way if other style developers could contribute with their opinions.
My suggestion: contribute towards improving the upgrade XSL stylesheet and help us either a) prove the fallback behavior isn't necessary, or b) show exactly why and how they are. Document this on the xbib list.
And CSL isn't just about Zotero and Mendeley. It's also now seeing development in PHP/Drupal, and has an implementation in Haskell (for use in part with the excellent pandoc tool); not to mention that I have a python version I periodically work on.
@Rintze - What about the "article" type redundancy in CSL 1.0? If you remove the fallback feature, the "article" type should become invalid.
I think you're right: if fallback behavior is scrapped, "article" can be scrapped as well ("article" is only used as a fallback type, in contrast to "book", which is the target of the Zotero "book" type, and "chapter", which is the target of the Zotero "bookSection" type). It probably wouldn't hurt to postpone the removal of "article" until CSL 1.1 though.
I've looked through the upgrade XSLT stylesheet and many things will have to be changed manually after the automatic conversion. I hope Zotero devs will give some advance notice before releasing CSL 1.0-compatible upgrade so we could test & revise CSLs to work with the new system.
I put the information of fallback behavior as defined in the Zotero source code in a slightly different format, which should be better suited for human consumption: http://docs.google.com/View?id=dg6h9k72_64hzmsmqgb
@erazlogo on the XSLT stylesheet: could you give specifics? I thought I had most things covered. You might want to post your comments to the xbiblio mailing list.
this is just not true, given the demands of styles like Chicago and Bluebook. Your example above would not work for manuscripts, for one.
@erazlogo: are you saying it's not possible to tweak the example macro I provided (which was just for illustration) so that it would work for stuff like manuscripts as well without using types?
In any case, I stand by the point that fallbacks are unnecessary; that the same logic can be easily represented using the conditional structure (and without having to include an exhaustive list of types there).
Also:
I've looked through the upgrade XSLT stylesheet and many things will have to be changed manually after the automatic conversion.
So rather than just note this, can you suggest improvements? I wrote the stylesheet awhile back, and I simply didn't consider this case. Might be hard to program, but maybe not.
@erazlogo: are you saying it's not possible to tweak the example macro I provided (which was just for illustration) so that it would work for stuff like manuscripts as well without using types?
No (see example below).
In any case, I stand by the point that fallbacks are unnecessary; that the same logic can be easily represented using the conditional structure (and without having to include an exhaustive list of types there).
Here is one of my title macros for Chicago in CSL 0.8: <macro name="title-note"> <choose> <if variable="title" match="none"> <text variable="genre"/> </if> <else-if type="book"> <text variable="title" font-style="italic"/> </else-if> <else> <text variable="title" quotes="true"/> </else> </choose> </macro>In CSL 1.0 the macro, using your example above and without the exhaustive list of types, will look like this: <macro name="title-note"> <choose> <if variable="title" match="none"> <text variable="genre"/> </if> <else-if variable="container-title"> <text variable="title" quotes="true"/> </else-if> <else-if type="manuscript thesis speech patent map"> <text variable="title" quotes="true"/> </else-if> <else> <text variable="title" font-style="italic"/> </else> </choose> </macro>An automatically converted macro with an exhaustive list of types will look like this: <macro name="title-note"> <choose> <if variable="title" match="none"> <text variable="genre"/> </if> <else-if type="book graphic song bill legal_case motion_picture report"> <text variable="title" font-style="italic"/> </else-if> <else> <text variable="title" quotes="true"/> </else> </choose> </macro> So, we can do without fallbacks, sure; but not without types. As you can see, the macro with the "exhaustive list of types" is actually simpler than the one without.
@erazlogo on the XSLT stylesheet: could you give specifics? I thought I had most things covered. You might want to post your comments to the xbiblio mailing list.
So rather than just note this, can you suggest improvements? I wrote the stylesheet awhile back, and I simply didn't consider this case. Might be hard to program, but maybe not.
Ok, I will look over and will start a new thread and post on the xbiblio list.
People don't typically believe me when I say this, but It's not generally necessary to use types to condition formatting.
There are people who overuse types and put them everywhere even if they could do without them as your example shows. That is not a problem of fallbacks, however. That is a problem of all types in general. Fallbacks (although you may have introduced them for a different reason) give developers the flexibility to use "supertypes" instead of individual types and thus can simplify the code like erazlogo showed. I agree that fallbacks are not necessary, however, I find them a great practical tool.
Now, for the role of types, it would be great if the styles could do without types entirely, but they cannot. I believe erazlogo's responses covered that well. On top of that I would like to re-iterate my point:
6/ Inability to access the native types in CSL 0.8 limits the level of compliance with some style guides and so makes the Zotero adoption difficult. Why not include more Zotero native types in the list of the standard CSL types or provide a native-type extension?
The point is about the other end of the mapping spectrum. In most cases, CSL style developers do not have a means to influence established style guides. My previous example of the requirement to distinguish between a web page (title in italics) and blog post (title in quotes) is a use case that is not supported at the moment. Is CSL going to be extended to include more native types?
My previous example of the requirement to distinguish between a web page (title in italics) and blog post (title in quotes) is a use case that is not supported at the moment.
Also, while preparing the fallback-behavior overview I mentioned above (http://docs.google.com/View?id=dg6h9k72_64hzmsmqgb), I noticed that there is a conflict that should be resolved if fallback behavior is going to be defined on a CSL-only level (without knowledge of the Zotero>CSL mapping): Zotero's "podcast" and "audioRecording" types are both mapped to "song", but "podcast" falls back to "article", whereas "audioRecording" falls back to "book".
@Rintze - Great work with the document! Could you also include:
Zotero type Document >> No CSL type >> Fallback type article
Another thought - Why don't we replace all CSL types with just the three fallback types and have some schema extension mechanism to support all native types? The current set of types is mapped from the Zotero native types arbitrarily anyway. Since no mapping guidelines are defined in the standard, various compliant processors may still produce incompatible output. It should be quite straightforward to define the mapping guidelines in the standard for a small set of fallback types. Moreover, this would be an incentive for CSL style developers to reduce the use of types as much as they can to make their CSL styles tool-independent. The upgrade would still require to migrate hundreds of styles though (unless you start with the native types identical to the current CSL types).
There are people who overuse types and put them everywhere even if they could do without them as your example shows. That is not a problem of fallbacks, however. That is a problem of all types in general. Fallbacks (although you may have introduced them for a different reason) give developers the flexibility to use "supertypes" instead of individual types ...
Since I'm the one that designed the fallback system, let me just say that the idea effectively was to use some syntactic sugar of sorts to describe generic, structural, characteristics. E.g. an "article" fallback was simply an item published serially in some periodical-like thing. You could describe the exact same thing by saying this "type" is anything with a "container-title" and a "volume" or "issue" number. A "chapter", by contrast, is something published in a one-off collection; e.g. it has a "container-title" and a "publisher" and no "volume" or "issue" number.
Keep in mind, too, that when I created this, there was no cs:choose element. Instead, there were separate template for each type, and the schema required templates for each of the fallback types.
Why not include more Zotero native types in the list of the standard CSL types or provide a native-type extension?
Zotero designed their list independently of the one I designed for CSL, and I don't agree with all their decisions. Extensions seems to me like a really bad idea, since it will encourage forking. Ideally, at some point, all CSL implementors come up with some ground rules on types and how they get extended, but I've been asking for this for literally years, and it's yet to see happen.
Another thought - Why don't we replace all CSL types with just the three fallback types and have some schema extension mechanism to support all native types?
Picking this up from previous reply, what exactly do you mean by the extension mechanism? Are you suggesting it be OK, for example, for styles created for one application use the term "journal-article" and styles for another use "JournalArticle"? How do you avoid that but still have the styles work across applications?
Keep in mind, too, that when I created this, there was no cs:choose element. Instead, there were separate template for each type, and the schema required templates for each of the fallback types.
Thanks for the link and explanation. I assume your concept of "class" describes what Zotero implements as the fallback type (monograph=book, part in a monographs=chapter, part in serials=article). If structural characteristics are clearly defined for these classes/fallbacks in the standard, keeping the classes in the standard would also effectively promote these concepts to CSL style developers (apart from the possible code simplification).
Are you suggesting it be OK, for example, for styles created for one application use the term "journal-article" and styles for another use "JournalArticle"?
Yes, with the caveat that by doing so the styles would only comply with a particular application dialect. This could encourage the use of structural characteristics instead, in case the style developers want to achieve full compliance with the standard. Consider that currently even if you have the same type names across applications they may still have different semantics (a "webpage" type may mean different things across applications).
On the other hand, a better approach may still be to keep the current CSL types and explicate their strong ontology in the standard (not just a controlled vocabulary of types).
I assume your concept of "class" describes what Zotero implements as the fallback type (monograph=book, part in a monographs=chapter, part in serials=article).
Exactly!
If structural characteristics are clearly defined for these classes/fallbacks in the standard, keeping the classes in the standard would also effectively promote these concepts to CSL style developers (apart from the possible code simplification).
Maybe; maybe not.
Am not entirely rejecting your argument here; I just think we have a few different balls in the air, and there may be other ways to ultimately address these issues (the most immediate being how to reliably update existing styles to 1.0). To be concrete, my short-to-long-term gaols are twofold: first, to improve the update XSL so conversion is fully automatic, and second, to make sure the online CSL wizard the Mendeley guys are working on is as good as it can be (so users don't have to worry about CSL syntax ;-)).
As I mentioned earlier, if we can document that the removal of the "class" types and fallbacks makes the first goal unattainable, then we can talk about adding it back in 1.1.
My suggestion: contribute towards improving the upgrade XSL stylesheet
my short-to-long-term gaols are twofold: first, to improve the update XSL so conversion is fully automatic
If fallback behavior is eliminated, updating update.xsl should be fairly trivial. For each conditional, you just have to replace each fallback type by all types that (in Zotero) fall back to that fallback type (as per the example above). The main reason I haven't made the required changes yet is that I'm waiting for consensus to either remove or keep fallback behavior for CSL 1.0.
While working on the upgrade path, could you also discuss this point ("blogPost" and "forumPost" mapping to "post-weblog" and "post" instead of "webpage", respectively)?
Upgrade Storage
Zotero Item Type >> CSL Type >> CSL Fallback Type
@bdarcus - If you remove the fallbacks, what do you plan to do about the existing styles that will thus get broken?
Also, will the mapping from Zotero Item Type to CSL Type stay as is? I do have a style that requires to distinguish between the Web page type and Blog post and I was hoping that would be possible one day.
Quick sampling through the available styles shows a heavy use of "book", "chapter" and "article" type conditions which may have been knowingly (or unknowingly for that matter) relying on the fallback feature.
My suggestion is:
1/ In CSL 1.0 keep the type processing logic as is in Zotero 2.0
2/ In CSL 1.1 modify csl-types.rnc (this is just a mock-up, hopefully backward-compatible and the part with the native types could be a Zotero-specific extension):
div {
cs-types =
cs-fallback-types
| cs-standard-types
| cs-native-types
}
div {
cs-fallback-types =
"article"
| "book"
| "chapter"
}
div {
cs-standard-types =
"article-journal"
| "article-magazine"
| "article-newspaper"
| "book-type"
| "book-section"
| ...
}
div {
cs-native-types =
"zot-artwork"
| "zot-audio-recording"
| "zot-bill"
| "zot-blog-post"
| "zot-book"
| "zot-book-section"
| ...
}
Some useful links:
Fallback mapping: https://www.zotero.org/trac/browser/extension/trunk/chrome/content/zotero/xpcom/csl.js?rev=5788#L1995
Older thread: http://forums.zotero.org/discussion/8989/resource-type-mapping-is-confusing/
The other problem I mentioned is about exposing the native types. Do I understand correctly from your example above that all Zotero item types, e.g. Computer Program, will be available in CSL? That would be great.
<if type="book motion_picture graphic report bill legal_case song motion_picture" match="any"/>
I expect Zotero won't switch in the new 1.0-compatible processor until the upgraded styles work correctly.
@bdarcus - Even when considering the CSL independence of Zotero...
1/ The fallback feature simplifies the citation style files and makes them easier to develop.
2/ The confusion about the fallback feature may be greatly reduced by using distinct names for types VS fallback types and by making the specification explicit about it.
3/ If a processor implementer cannot comply with the standard, they should simply document the incompatibility. In this particular case, the fallback feature implementation should be straightforward and the Zotero source code is available to everyone, including Mendeley and Pandoc.
4/ The mapping from the native types to CSL types is, too, a kind of a fallback layer.
Considering Zotero...
5/ Zotero is not in the beta stage (cf. Mendeley) and a smooth upgrade path will be crucial.
6/ Inability to access the native types in CSL 0.8 limits the level of compliance with some style guides and so makes the Zotero adoption difficult. Why not include more Zotero native types in the list of the standard CSL types or provide a native-type extension?
I hope there is still some space for a discussion on this toward the CSL 1.0 implementation or CSL 1.1 specification. Maybe the case could be made stronger either way if other style developers could contribute with their opinions.
<macro name="title">
<choose>
<if variable="container-title">
<text variable="title" quotes="true"/>
</if>
<else>
<text variable="title"/>
<else>
</choose>
</macro>
People don't typically believe me when I say this, but It's not generally necessary to use types to condition formatting. My suggestion: contribute towards improving the upgrade XSL stylesheet and help us either a) prove the fallback behavior isn't necessary, or b) show exactly why and how they are. Document this on the xbib list.
And CSL isn't just about Zotero and Mendeley. It's also now seeing development in PHP/Drupal, and has an implementation in Haskell (for use in part with the excellent pandoc tool); not to mention that I have a python version I periodically work on.
@bdarcus: what is the current selection of CSL item types based on? The Bibliographic Ontology? (http://bibliontology.com/)
@erazlogo on the XSLT stylesheet: could you give specifics? I thought I had most things covered. You might want to post your comments to the xbiblio mailing list.
In any case, I stand by the point that fallbacks are unnecessary; that the same logic can be easily represented using the conditional structure (and without having to include an exhaustive list of types there).
Also: So rather than just note this, can you suggest improvements? I wrote the stylesheet awhile back, and I simply didn't consider this case. Might be hard to program, but maybe not.
<macro name="title-note">In CSL 1.0 the macro, using your example above and without the exhaustive list of types, will look like this:<choose>
<if variable="title" match="none">
<text variable="genre"/>
</if>
<else-if type="book">
<text variable="title" font-style="italic"/>
</else-if>
<else>
<text variable="title" quotes="true"/>
</else>
</choose>
</macro>
<macro name="title-note">An automatically converted macro with an exhaustive list of types will look like this:<choose>
<if variable="title" match="none">
<text variable="genre"/>
</if>
<else-if variable="container-title">
<text variable="title" quotes="true"/>
</else-if>
<else-if type="manuscript thesis speech patent map">
<text variable="title" quotes="true"/>
</else-if>
<else>
<text variable="title" font-style="italic"/>
</else>
</choose>
</macro>
<macro name="title-note"><choose>
<if variable="title" match="none">
<text variable="genre"/>
</if>
<else-if type="book graphic song bill legal_case motion_picture report">
<text variable="title" font-style="italic"/>
</else-if>
<else>
<text variable="title" quotes="true"/>
</else>
</choose>
</macro>
So, we can do without fallbacks, sure; but not without types. As you can see, the macro with the "exhaustive list of types" is actually simpler than the one without. Ok, I will look over and will start a new thread and post on the xbiblio list.
Now, for the role of types, it would be great if the styles could do without types entirely, but they cannot. I believe erazlogo's responses covered that well. On top of that I would like to re-iterate my point: The point is about the other end of the mapping spectrum. In most cases, CSL style developers do not have a means to influence established style guides. My previous example of the requirement to distinguish between a web page (title in italics) and blog post (title in quotes) is a use case that is not supported at the moment. Is CSL going to be extended to include more native types?
Also, while preparing the fallback-behavior overview I mentioned above (http://docs.google.com/View?id=dg6h9k72_64hzmsmqgb), I noticed that there is a conflict that should be resolved if fallback behavior is going to be defined on a CSL-only level (without knowledge of the Zotero>CSL mapping): Zotero's "podcast" and "audioRecording" types are both mapped to "song", but "podcast" falls back to "article", whereas "audioRecording" falls back to "book".
Zotero type Document >> No CSL type >> Fallback type article
Another thought - Why don't we replace all CSL types with just the three fallback types and have some schema extension mechanism to support all native types? The current set of types is mapped from the Zotero native types arbitrarily anyway. Since no mapping guidelines are defined in the standard, various compliant processors may still produce incompatible output. It should be quite straightforward to define the mapping guidelines in the standard for a small set of fallback types. Moreover, this would be an incentive for CSL style developers to reduce the use of types as much as they can to make their CSL styles tool-independent. The upgrade would still require to migrate hundreds of styles though (unless you start with the native types identical to the current CSL types).
I described this in this document.
Keep in mind, too, that when I created this, there was no cs:choose element. Instead, there were separate template for each type, and the schema required templates for each of the fallback types. Zotero designed their list independently of the one I designed for CSL, and I don't agree with all their decisions. Extensions seems to me like a really bad idea, since it will encourage forking. Ideally, at some point, all CSL implementors come up with some ground rules on types and how they get extended, but I've been asking for this for literally years, and it's yet to see happen.
On the other hand, a better approach may still be to keep the current CSL types and explicate their strong ontology in the standard (not just a controlled vocabulary of types).
Am not entirely rejecting your argument here; I just think we have a few different balls in the air, and there may be other ways to ultimately address these issues (the most immediate being how to reliably update existing styles to 1.0). To be concrete, my short-to-long-term gaols are twofold: first, to improve the update XSL so conversion is fully automatic, and second, to make sure the online CSL wizard the Mendeley guys are working on is as good as it can be (so users don't have to worry about CSL syntax ;-)).
As I mentioned earlier, if we can document that the removal of the "class" types and fallbacks makes the first goal unattainable, then we can talk about adding it back in 1.1.