connection refused when processing with zrst2pdf
Created by: mcepl
When running zrst2pdf
with https://gitlab.com/mcepl/christian_unity/blob/master/ut_unum_sint.rst (on rhel-7 so with python 2.7 and pyparsing 1.5.7):
matej@mitmanek: christian_unity (master *)$ zrst2pdf ut_unum_sint.rst
Traceback (most recent call last):
File "/home/matej/.local/bin/zrst2pdf", line 8, in <module>
execfile(__file__)
File "/home/matej/archiv/knihovna/repos/zot4rst/bin/zrst2pdf", line 12, in <module>
load_entry_point('rst2pdf', 'console_scripts', 'rst2pdf')()
File "/home/matej/.local/lib/python2.7/site-packages/rst2pdf-0.93.dev-py2.7.egg/rst2pdf/createpdf.py", line 1495, in main
compressed=options.compressed)
File "/home/matej/.local/lib/python2.7/site-packages/rst2pdf-0.93.dev-py2.7.egg/rst2pdf/createpdf.py", line 540, in createPdf
settings_overrides=settings_overrides)
File "/usr/lib/python2.7/site-packages/docutils/core.py", line 479, in publish_doctree
output = pub.publish(enable_exit_status=enable_exit_status)
File "/usr/lib/python2.7/site-packages/docutils/core.py", line 218, in publish
self.apply_transforms()
File "/usr/lib/python2.7/site-packages/docutils/core.py", line 199, in apply_transforms
self.document.transformer.apply_transforms()
File "/usr/lib/python2.7/site-packages/docutils/transforms/__init__.py", line 171, in apply_transforms
transform.apply(**kwargs)
File "/home/matej/archiv/knihovna/repos/zot4rst/xciterst/directives.py", line 27, in apply
self.startnode.replace_self(xciterst.citeproc.generate_rest_bibliography())
File "/home/matej/archiv/knihovna/repos/zot4rst/xciterst/__init__.py", line 45, in generate_rest_bibliography
bibdata = self.citeproc_process(clusters)[1]
File "/home/matej/archiv/knihovna/repos/zot4rst/zot4rst/__init__.py", line 31, in citeproc_process
f = urllib2.urlopen(req)
File "/usr/lib64/python2.7/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib64/python2.7/urllib2.py", line 431, in open
response = self._open(req, data)
File "/usr/lib64/python2.7/urllib2.py", line 449, in _open
'_open', req)
File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/usr/lib64/python2.7/urllib2.py", line 1244, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "/usr/lib64/python2.7/urllib2.py", line 1214, in do_open
raise URLError(err)
urllib2.URLError: <urlopen error [Errno 111] Connection refused>
Any idea what's wrong, please?
Thank you for the great tool!
P.S.: Actually, the same error happens when processing example.rst
.
Imported comments:
By mcepl on 2015-07-29 19:43:34+00:00
Hmm,
matej@mitmanek: christian_unity (master %)$ telnet localhost 23119
Trying 127.0.0.1...
telnet: connect to address 127.0.0.1: Connection refused
Any idea, why zotxt is not running? Using Firefox 38.1.0 on RHEL-7.
By mcepl on 2015-07-29 19:56:26+00:00
OK, so I needed to restart Firefox to have zotxt server working. Now
when I open in Firefox http://localhost:23119/zotxt/items?easykey=mcclung:1987father
I get a reasonably looking JSON (actually http://jsonlint.com/ agrees that is a valid JSON), but when I run zrst4pdf
I get this:
matej@mitmanek: christian_unity (master *%)$ zrst2pdf ut_unum_sint.rst
Traceback (most recent call last):
File "/home/matej/.local/bin/zrst2pdf", line 8, in <module>
execfile(__file__)
File "/home/matej/archiv/knihovna/repos/zot4rst/bin/zrst2pdf", line 12, in <module>
load_entry_point('rst2pdf', 'console_scripts', 'rst2pdf')()
File "/home/matej/.local/lib/python2.7/site-packages/rst2pdf-0.93.dev-py2.7.egg/rst2pdf/createpdf.py", line 1495, in main
compressed=options.compressed)
File "/home/matej/.local/lib/python2.7/site-packages/rst2pdf-0.93.dev-py2.7.egg/rst2pdf/createpdf.py", line 540, in createPdf
settings_overrides=settings_overrides)
File "/usr/lib/python2.7/site-packages/docutils/core.py", line 479, in publish_doctree
output = pub.publish(enable_exit_status=enable_exit_status)
File "/usr/lib/python2.7/site-packages/docutils/core.py", line 217, in publish
self.settings)
File "/usr/lib/python2.7/site-packages/docutils/readers/__init__.py", line 72, in read
self.parse()
File "/usr/lib/python2.7/site-packages/docutils/readers/__init__.py", line 78, in parse
self.parser.parse(self.input, document)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/__init__.py", line 172, in parse
self.statemachine.run(inputlines, document, inliner=self.inliner)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 170, in run
input_source=document['source'])
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 239, in run
context, state, transitions)
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 460, in check_line
return method(match, context, next_state)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2726, in underline
self.section(title, source, style, lineno - 1, messages)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 327, in section
self.new_subsection(title, lineno, messages)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 395, in new_subsection
node=section_node, match_titles=True)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
node=node, match_titles=match_titles)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 195, in run
results = StateMachineWS.run(self, input_lines, input_offset)
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 239, in run
context, state, transitions)
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 460, in check_line
return method(match, context, next_state)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2726, in underline
self.section(title, source, style, lineno - 1, messages)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 327, in section
self.new_subsection(title, lineno, messages)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 395, in new_subsection
node=section_node, match_titles=True)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
node=node, match_titles=match_titles)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 195, in run
results = StateMachineWS.run(self, input_lines, input_offset)
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 239, in run
context, state, transitions)
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 460, in check_line
return method(match, context, next_state)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2726, in underline
self.section(title, source, style, lineno - 1, messages)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 327, in section
self.new_subsection(title, lineno, messages)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 395, in new_subsection
node=section_node, match_titles=True)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
node=node, match_titles=match_titles)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 195, in run
results = StateMachineWS.run(self, input_lines, input_offset)
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 239, in run
context, state, transitions)
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 460, in check_line
return method(match, context, next_state)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2740, in text
paragraph, literalnext = self.paragraph(lines, startline)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 418, in paragraph
textnodes, messages = self.inline_text(text, lineno)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 427, in inline_text
return self.inliner.parse(text, lineno, self.memo, self.parent)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 513, in parse
lineno)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 776, in interpreted_or_phrase_ref
lineno)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 856, in interpreted
nodes, messages2 = role_fn(role, rawsource, text, lineno, self)
File "/home/matej/archiv/knihovna/repos/zot4rst/xciterst/roles.py", line 55, in cite_role
[first_cluster, second_cluster] = CiteParser().parse(text)
File "/home/matej/archiv/knihovna/repos/zot4rst/xciterst/parser.py", line 135, in parse
raw = topCite.parseString(what, True)
File "/home/matej/.local/lib/python2.7/site-packages/pyparsing.py", line 1006, in parseString
raise exc
pyparsing.ParseException: Expected end of text (at char 7), (line:1, col:8)
By mcepl on 2015-07-29 19:58:03+00:00
My library exported as CSL JSON is http://mcepl.fedorapeople.org/tmp/My%20Library.json
By mcepl on 2015-07-29 19:59:33+00:00
Must all keys be US-ASCII? I have keys like @halík:2011tomáš
.
By mcepl on 2015-07-29 22:45:14+00:00
When I added logging
module and added logging of topCite
and what
just before I got this (I am afraid that topCite makes me scared):
INFO:add_extensions:Importing extension module 'vectorpdf'
INFO:get_language_available:Language 'en_US' not supported by Docutils, using 'en' instead.
INFO:__init__:Using stylesheets:
INFO:__init__:FontPath:['/home/matej/.local/lib/python2.7/site-packages/rst2pdf-0.93.dev-py2.7.egg/rst2pdf/fonts', '.']
INFO:__init__:StylePath:['/home/matej/.local/lib/python2.7/site-packages/rst2pdf-0.93.dev-py2.7.egg/rst2pdf/styles', '/home/matej/.rst2pdf/styles', '.']
INFO:get_language_available:Language 'en_US' not supported by Docutils, using 'en' instead.
DEBUG:parse:topCite = {{{{{{"[" {{{{{["-"] "@"} Re:('[\\w-]+')} | {{{{{{"**" {Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}...} "**"} | {{"*" {Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}...} "*"}} | Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}}... {{["-"] "@"} Re:('[\\w-]+')}}} [{{[","] {Re:('[\\w.,\\\'\\"\\(\\)</>-]*[0-9][\\w.,\\\'\\"\\(\\)</>-]*')}...} ^ {{[","] [Re:('[\\w.,\\\'\\"\\(\\)</>-]+')]} {Re:('[\\w.,\\\'\\"\\(\\)</>-]*[0-9][\\w.,\\\'\\"\\(\\)</>-]*')}...}}]} [{{{{{"**" {Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}...} "**"} | {{"*" {Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}...} "*"}} | Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}}...]}} [{";" {{{{{["-"] "@"} Re:('[\\w-]+')} | {{{{{{"**" {Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}...} "**"} | {{"*" {Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}...} "*"}} | Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}}... {{["-"] "@"} Re:('[\\w-]+')}}} [{{[","] {Re:('[\\w.,\\\'\\"\\(\\)</>-]*[0-9][\\w.,\\\'\\"\\(\\)</>-]*')}...} ^ {{[","] [Re:('[\\w.,\\\'\\"\\(\\)</>-]+')]} {Re:('[\\w.,\\\'\\"\\(\\)</>-]*[0-9][\\w.,\\\'\\"\\(\\)</>-]*')}...}}]} [{{{{{"**" {Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}...} "**"} | {{"*" {Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}...} "*"}} | Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}}...]}}]...} "]"} ^ {{{["-"] "@"} Re:('[\\w-]+')} {{{{"[" {{[","] {Re:('[\\w.,\\\'\\"\\(\\)</>-]*[0-9][\\w.,\\\'\\"\\(\\)</>-]*')}...} ^ {{[","] [Re:('[\\w.,\\\'\\"\\(\\)</>-]+')]} {Re:('[\\w.,\\\'\\"\\(\\)</>-]*[0-9][\\w.,\\\'\\"\\(\\)</>-]*')}...}}} [{{{{{"**" {Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}...} "**"} | {{"*" {Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}...} "*"}} | Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}}...]} [{";" {{{{{["-"] "@"} Re:('[\\w-]+')} | {{{{{{"**" {Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}...} "**"} | {{"*" {Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}...} "*"}} | Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}}... {{["-"] "@"} Re:('[\\w-]+')}}} [{{[","] {Re:('[\\w.,\\\'\\"\\(\\)</>-]*[0-9][\\w.,\\\'\\"\\(\\)</>-]*')}...} ^ {{[","] [Re:('[\\w.,\\\'\\"\\(\\)</>-]+')]} {Re:('[\\w.,\\\'\\"\\(\\)</>-]*[0-9][\\w.,\\\'\\"\\(\\)</>-]*')}...}}]} [{{{{{"**" {Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}...} "**"} | {{"*" {Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}...} "*"}} | Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}}...]}}]...} "]"}}} ^ {{{["-"] "@"} Re:('[\\w-]+')} {{{"[" {{{{{["-"] "@"} Re:('[\\w-]+')} | {{{{{{"**" {Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}...} "**"} | {{"*" {Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}...} "*"}} | Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}}... {{["-"] "@"} Re:('[\\w-]+')}}} [{{[","] {Re:('[\\w.,\\\'\\"\\(\\)</>-]*[0-9][\\w.,\\\'\\"\\(\\)</>-]*')}...} ^ {{[","] [Re:('[\\w.,\\\'\\"\\(\\)</>-]+')]} {Re:('[\\w.,\\\'\\"\\(\\)</>-]*[0-9][\\w.,\\\'\\"\\(\\)</>-]*')}...}}]} [{{{{{"**" {Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}...} "**"} | {{"*" {Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}...} "*"}} | Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}}...]}} [{";" {{{{{["-"] "@"} Re:('[\\w-]+')} | {{{{{{"**" {Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}...} "**"} | {{"*" {Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}...} "*"}} | Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}}... {{["-"] "@"} Re:('[\\w-]+')}}} [{{[","] {Re:('[\\w.,\\\'\\"\\(\\)</>-]*[0-9][\\w.,\\\'\\"\\(\\)</>-]*')}...} ^ {{[","] [Re:('[\\w.,\\\'\\"\\(\\)</>-]+')]} {Re:('[\\w.,\\\'\\"\\(\\)</>-]*[0-9][\\w.,\\\'\\"\\(\\)</>-]*')}...}}]} [{{{{{"**" {Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}...} "**"} | {{"*" {Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}...} "*"}} | Re:('[\\w.,\\\'\\"\\(\\)</>-]+')}}...]}}]...} "]"}}} ^ {{["-"] "@"} Re:('[\\w-]+')}}
DEBUG:parse:what = @martin:2012acts
When running http://localhost:23119/zotxt/items?easykey=martin:2012acts I get a reasonable JSON reply.
By egh on 2015-08-02 06:18:24+00:00
Hi Matěj - Thanks for the report! The zotxt extension supports non-ascii keys but it's quite likely that zot4rst doesn't. I am mostly using pandoc-zotxt now.
The regex looks horrible, but I promise the underlying parser grammar is pretty simple. :)
I'll have a look at supporting non-ascii keys. It should be pretty easy.
By egh on 2015-08-02 07:03:29+00:00
I believe that I have fixed this - the problem was that I was not allowing :
in the grammar for citekeys.
Please note that while I'll try to fix issues, I'm not really doing development on zot4rst, though development continues on zotxt. Even though reStructuredText is superior, I have given in and use markdown now.
By mcepl on 2015-08-03 06:41:03+00:00
matej@mitmanek: christian_unity (master *%)$ zrst2rst ut_unum_sint.rst >ut_unum_sint-filled.rst
Traceback (most recent call last):
File "/home/matej/.local/bin/zrst2rst", line 6, in <module>
exec(compile(open(__file__).read(), __file__, 'exec'))
File "/home/matej/archiv/knihovna/repos/zot4rst/bin/zrst2rst", line 771, in <module>
doc=docutils.core.publish_doctree(input)
File "/usr/lib/python2.7/site-packages/docutils/core.py", line 479, in publish_doctree
output = pub.publish(enable_exit_status=enable_exit_status)
File "/usr/lib/python2.7/site-packages/docutils/core.py", line 217, in publish
self.settings)
File "/usr/lib/python2.7/site-packages/docutils/readers/__init__.py", line 72, in read
self.parse()
File "/usr/lib/python2.7/site-packages/docutils/readers/__init__.py", line 78, in parse
self.parser.parse(self.input, document)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/__init__.py", line 172, in parse
self.statemachine.run(inputlines, document, inliner=self.inliner)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 170, in run
input_source=document['source'])
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 239, in run
context, state, transitions)
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 460, in check_line
return method(match, context, next_state)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2726, in underline
self.section(title, source, style, lineno - 1, messages)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 327, in section
self.new_subsection(title, lineno, messages)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 395, in new_subsection
node=section_node, match_titles=True)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
node=node, match_titles=match_titles)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 195, in run
results = StateMachineWS.run(self, input_lines, input_offset)
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 239, in run
context, state, transitions)
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 460, in check_line
return method(match, context, next_state)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2726, in underline
self.section(title, source, style, lineno - 1, messages)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 327, in section
self.new_subsection(title, lineno, messages)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 395, in new_subsection
node=section_node, match_titles=True)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
node=node, match_titles=match_titles)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 195, in run
results = StateMachineWS.run(self, input_lines, input_offset)
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 239, in run
context, state, transitions)
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 460, in check_line
return method(match, context, next_state)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2726, in underline
self.section(title, source, style, lineno - 1, messages)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 327, in section
self.new_subsection(title, lineno, messages)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 395, in new_subsection
node=section_node, match_titles=True)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
node=node, match_titles=match_titles)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 195, in run
results = StateMachineWS.run(self, input_lines, input_offset)
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 239, in run
context, state, transitions)
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 460, in check_line
return method(match, context, next_state)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2740, in text
paragraph, literalnext = self.paragraph(lines, startline)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 418, in paragraph
textnodes, messages = self.inline_text(text, lineno)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 427, in inline_text
return self.inliner.parse(text, lineno, self.memo, self.parent)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 513, in parse
lineno)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 776, in interpreted_or_phrase_ref
lineno)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 856, in interpreted
nodes, messages2 = role_fn(role, rawsource, text, lineno, self)
File "/home/matej/archiv/knihovna/repos/zot4rst/xciterst/roles.py", line 55, in cite_role
[first_cluster, second_cluster] = CiteParser().parse(text)
File "/home/matej/archiv/knihovna/repos/zot4rst/xciterst/parser.py", line 135, in parse
raw = topCite.parseString(what, True)
File "/home/matej/.local/lib/python2.7/site-packages/pyparsing.py", line 1006, in parseString
raise exc
pyparsing.ParseException: Expected end of text (at char 25), (line:1, col:26)
matej@mitmanek: christian_unity (master *%)$
Would it be possible to add a little bit of diagnostics (e.g., which key was being processed). When I comment out .. zotero-setup
, .. default-role
, and .. bibliography
I get perfectly working document.
Concerning giving up on the reStructuredText itself. I don’t understand. If it works for you, then why to give up? And some parts of the Markdown (significant trailing whitespace!) just make me believe that MD is good just enough for short comments in the issue trackers, but my real work should be in something more robust and thought-through.
By mcepl on 2015-08-03 06:44:53+00:00
This happens even when I comment out all non-ASCII keys (replace @something
with CITE something so I guess it shouldn't get recognized as a key anymore).
By egh on 2015-08-03 07:13:57+00:00
Hi Matěj - The problem was not non-ascii characters, it was the :
in a key. Have you upgraded to the source version? I haven't pushed it to pypi yet.
I agree about reST over markdown, but I've found pandoc to be good enough for my usage. In any case, I'm not trying to convince you, I just wanted to let you know that I may not be as active on this project as I might be if I was using it full time. Thanks!
By egh on 2015-08-03 07:22:42+00:00
Oh, I see from the backtrace that you are running from the local repo.
zot4rst will cite everything within the
:xcite:`...`
so if you use
:xcite:`CITE something`
It will not be parsed correctly.
I have pushed a fix to throw an exception with the text of the citation when there is an error parsing.
Thanks!
By mcepl on 2015-08-03 07:41:26+00:00
:xcite:`CITE something`
I have removed even backquotes.
I have pushed a fix to throw an exception with the text of the citation when there is an error parsing.
Where? BTW, what is the canonical source of zot4rst
... this repo or the one at GitHub (both have issue trackers switched on)?
By mcepl on 2015-08-03 15:15:46+00:00
I have added some debugging myself (see https://gitlab.com/mcepl/zot4rst/tree/more_debugging_parser) and it seems that zot4rst
managed perfectly well the first two citations (`@martin:2012acts`
and `@anonymous:2015aelia`
), but crashed on the third one (`@anonymous:encyclopaedia [“Antioch”]`
... here the "Antioch" is quoted by the curly quotes, but the same crash happens with the normal "inch signs"). However, when looking at https://bitbucket.org/egh/zot4rst/raw/f674b0611cd917d0bce9191c364a0861d34cde1b/example/example.rst it seems to me that it is exactly what's expected. Isn't it?
Next crash happens on `@jenkins:2009lost [p. 39]`
and see @mcclung:1987father`
... again, what's wrong with these?
matej@mitmanek: christian_unity (master *%)$ zrst2rst ut_unum_sint.rst >ut_unum_sint-filled.rst
DEBUG:cite_role:parsing text = @martin:2012acts
DEBUG:cite_role:parsing text = @anonymous:2015aelia
DEBUG:cite_role:parsing text = @anonymous:encyclopaedia [“Antioch”]
WARNING:parse:parsing string failed: string = @anonymous:encyclopaedia [“Antioch”]
Traceback (most recent call last):
File "/home/matej/.local/bin/zrst2rst", line 6, in <module>
exec(compile(open(__file__).read(), __file__, 'exec'))
File "/home/matej/archiv/knihovna/repos/zot4rst/bin/zrst2rst", line 771, in <module>
doc=docutils.core.publish_doctree(input)
File "/usr/lib/python2.7/site-packages/docutils/core.py", line 479, in publish_doctree
output = pub.publish(enable_exit_status=enable_exit_status)
File "/usr/lib/python2.7/site-packages/docutils/core.py", line 217, in publish
self.settings)
File "/usr/lib/python2.7/site-packages/docutils/readers/__init__.py", line 72, in read
self.parse()
File "/usr/lib/python2.7/site-packages/docutils/readers/__init__.py", line 78, in parse
self.parser.parse(self.input, document)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/__init__.py", line 172, in parse
self.statemachine.run(inputlines, document, inliner=self.inliner)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 170, in run
input_source=document['source'])
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 239, in run
context, state, transitions)
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 460, in check_line
return method(match, context, next_state)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2726, in underline
self.section(title, source, style, lineno - 1, messages)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 327, in section
self.new_subsection(title, lineno, messages)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 395, in new_subsection
node=section_node, match_titles=True)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
node=node, match_titles=match_titles)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 195, in run
results = StateMachineWS.run(self, input_lines, input_offset)
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 239, in run
context, state, transitions)
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 460, in check_line
return method(match, context, next_state)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2726, in underline
self.section(title, source, style, lineno - 1, messages)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 327, in section
self.new_subsection(title, lineno, messages)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 395, in new_subsection
node=section_node, match_titles=True)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
node=node, match_titles=match_titles)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 195, in run
results = StateMachineWS.run(self, input_lines, input_offset)
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 239, in run
context, state, transitions)
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 460, in check_line
return method(match, context, next_state)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2726, in underline
self.section(title, source, style, lineno - 1, messages)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 327, in section
self.new_subsection(title, lineno, messages)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 395, in new_subsection
node=section_node, match_titles=True)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
node=node, match_titles=match_titles)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 195, in run
results = StateMachineWS.run(self, input_lines, input_offset)
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 239, in run
context, state, transitions)
File "/usr/lib/python2.7/site-packages/docutils/statemachine.py", line 460, in check_line
return method(match, context, next_state)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 2740, in text
paragraph, literalnext = self.paragraph(lines, startline)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 418, in paragraph
textnodes, messages = self.inline_text(text, lineno)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 427, in inline_text
return self.inliner.parse(text, lineno, self.memo, self.parent)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 513, in parse
lineno)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 776, in interpreted_or_phrase_ref
lineno)
File "/usr/lib/python2.7/site-packages/docutils/parsers/rst/states.py", line 856, in interpreted
nodes, messages2 = role_fn(role, rawsource, text, lineno, self)
File "/home/matej/archiv/knihovna/repos/zot4rst/xciterst/roles.py", line 59, in cite_role
[first_cluster, second_cluster] = CiteParser().parse(text)
File "/home/matej/archiv/knihovna/repos/zot4rst/xciterst/parser.py", line 140, in parse
raise ex
pyparsing.ParseException: Expected end of text (at char 25), (line:1, col:26)
matej@mitmanek: christian_unity (master *%)$
By egh on 2015-08-04 01:54:17+00:00
github & bitbucket should be in sync. I am trying to move to gitlab to support free software, but I like to maintain mirrors.
Generally citations should be enclosed in square brackets. I know this is a little weird but it is the pandoc style. (it makes more sense in pandoc, where the citations do not need to be offset by a role, as in reST.) The plain @citation
style is for where you want the citation to look like Doe (2006)
instead of (Doe 2006)
(in Chicago author date style). See http://pandoc.org/demo/example19/Extension-citations.html for a little more information.
So you might want to rewrite see @mcclung:1987father
as [see @mcclung:1987father]
or move see
out of the xcite
role.
@jenkins:2009lost [p. 39]
works for me, though, again, you might want to use [@jenkins:2009lost p. 39]
instead.
@anonymous:encyclopaedia ["Antioch”]
will not work because "Antioch"
is not a locator but a suffix - that is, @foo [p. 10]
is allowed but @foo [bar]
is not. However [@foo bar]
should be allowed.
I think the non-ascii in “Antioch”
is causing an issue that I will look into.
By egh on 2015-08-04 02:25:43+00:00
I have pushed a fix for unicode quotation marks in suffixes.
By mcepl on 2015-08-04 06:52:44+00:00
Even this most simple testing document throws again the connection refused error (but this time when checking the URL in browser I truly get "Endpoint does not support method"):
matej@mitmanek: christian_unity (master *%)$ zrst2rst test_cite.rst >test_cite-filled.rst
WARNING:citeproc_process:cannot open URL http://localhost:23119/zotxt/bibliography
Traceback (most recent call last):
File "/home/matej/.local/bin/zrst2rst", line 6, in <module>
exec(compile(open(__file__).read(), __file__, 'exec'))
File "/home/matej/archiv/knihovna/repos/zot4rst/bin/zrst2rst", line 771, in <module>
doc=docutils.core.publish_doctree(input)
File "/usr/lib/python2.7/site-packages/docutils/core.py", line 479, in publish_doctree
output = pub.publish(enable_exit_status=enable_exit_status)
File "/usr/lib/python2.7/site-packages/docutils/core.py", line 218, in publish
self.apply_transforms()
File "/usr/lib/python2.7/site-packages/docutils/core.py", line 199, in apply_transforms
self.document.transformer.apply_transforms()
File "/usr/lib/python2.7/site-packages/docutils/transforms/__init__.py", line 171, in apply_transforms
transform.apply(**kwargs)
File "/home/matej/archiv/knihovna/repos/zot4rst/xciterst/directives.py", line 132, in apply
newnode = xciterst.citeproc.get_citation(cite_cluster)
File "/home/matej/archiv/knihovna/repos/zot4rst/xciterst/__init__.py", line 58, in get_citation
self.cache_citations()
File "/home/matej/archiv/knihovna/repos/zot4rst/xciterst/__init__.py", line 54, in cache_citations
html = self.citeproc_process(clusters)[0]
File "/home/matej/archiv/knihovna/repos/zot4rst/zot4rst/__init__.py", line 41, in citeproc_process
f = urllib2.urlopen(req)
File "/usr/lib64/python2.7/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib64/python2.7/urllib2.py", line 437, in open
response = meth(req, response)
File "/usr/lib64/python2.7/urllib2.py", line 550, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib64/python2.7/urllib2.py", line 475, in error
return self._call_chain(*args)
File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/usr/lib64/python2.7/urllib2.py", line 558, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 400: Bad Request
matej@mitmanek: christian_unity (master *%)$
By mcepl on 2015-08-04 06:55:49+00:00
Exported library as CSL JSON.
By mcepl on 2015-08-04 07:14:02+00:00
Before you make that gitlab repository (notice, that gitlab can import whole repository from BitBucket including all issues, pull requests and other metadata), I've made a fork here and this pull request https://bitbucket.org/egh/zot4rst/pull-requests/1/
By egh on 2015-08-04 16:02:33+00:00
Thanks! Pasting http://localhost:23119/zotxt/bibliography
in the browser should say Endpoint does not support method
because the endpoint only uses POST
.
By egh on 2015-08-04 16:19:22+00:00
I'll try to figure out the issue that you are seeing.
By mcepl on 2015-08-18 12:03:02+00:00
Is there actually somewhere a documentation for http://localhost:23119/zotxt/bibliography
endpoint? I cannot find it on https://bitbucket.org/egh/zotxt anywhere?
I guess, with dependence on pandoc
(and pandoc-citeproc
) you don't care about formatting itself in zotxt
, right? Should I just give up any hope zotxt
would format bibliography for me and incorporate call to citeproc-py somewhere in zot4rst
?
By egh on 2015-08-19 19:46:21+00:00
Sorry, I was on vacation last week so haven't had a chance to look into this further.
the bibliography
endpoint is not documented, but it's pretty straightforward and should work.
By mcepl on 2015-08-19 19:57:09+00:00
And could I get some example how bibliography
endpoint should work? Some simple script perhaps?
By egh on 2015-08-19 20:09:33+00:00
Hope that helps.
By egh on 2015-08-19 20:11:27+00:00
Also see: http://gsl-nagoya-u.net/http/pub/citeproc-doc.html#citation-data-object for info on the structure of the citationItems
By mcepl on 2015-08-20 14:51:10+00:00
OK, I don’t do Ruby very well (or at all), so I have created this simple Python script (more or less pasting here so that there is a documentation):
#!/usr/bin/env python
import json
import logging
logging.basicConfig(format='%(levelname)s:%(funcName)s:%(message)s',
level=logging.DEBUG)
import urllib2
rdata = {
"styleId": "chicago-author-date",
"citationGroups": [{
"citationItems": [{
"easyKey": 'martin:2012acts'
}],
"properties": {
"noteIndex": 0
}
}]
}
req = urllib2.Request('http://localhost:23119/zotxt/bibliography')
req.add_header('Content-Type', 'application/json')
handler = urllib2.HTTPHandler(debuglevel=2)
opener = urllib2.build_opener(handler)
res = opener.open(req, data=json.dumps(rdata))
logging.debug('res = getcode %s\n', res.getcode())
logging.debug('data:\n%s', res.read())
And it seems to work right:
matej@mitmanek: zot4rst (master %)$ python play_bibliography.py
send: 'POST /zotxt/bibliography HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 139\r\nHost: localhost:23119\r\nAccept: application/json\r\nUser-Agent: Python-urllib/2.7\r\nConnection: close\r\nContent-Type: application/json\r\n\r\n{"citationGroups": [{"citationItems": [{"easyKey": "martin:2012acts"}], "properties": {"noteIndex": 0}}], "styleId": "chicago-author-date"}'
reply: 'HTTP/1.0 200 OK\r\n'
header: X-Zotero-Version: 4.0.28.1
header: X-Zotero-Connector-API-Version: 2
header: Content-Type: application/json; charset=UTF-8
DEBUG:<module>:res = getcode 200
DEBUG:<module>:data:
{
"bibliography": [
{
"maxoffset": 0,
"entryspacing": 0,
"linespacing": 1,
"second-field-align": false,
"entry_ids": [
[
"125"
]
],
"bibliography_errors": [],
"done": false,
"hangingindent": 2,
"bibstart": "<div class=\"csl-bib-body\">\n",
"bibend": "</div>"
},
[
" <div class=\"csl-entry\">Martin, Dale B. 2012. <i>The Acts of the Apostles</i>. Vol. 10. 26 vols. Open Yale Courses | Introduction to the New Testament History and Literature. Yale University. <a href=\"http://oyc.yale.edu/religious-studies/rlst-152/lecture-10\">http://oyc.yale.edu/religious-studies/rlst-152/lecture-10</a>.</div>\n"
]
],
"citationClusters": [
"(Martin 2012)"
]
}
By mcepl on 2015-08-25 16:48:15+00:00
The problem seems to be with the multiple citationItems
in one citationGroups
with different easyKey
(if they are same, there is no problem). So, if I modify my script to:
#!/usr/bin/env python
import json
import logging
logging.basicConfig(format='%(levelname)s:%(funcName)s:%(message)s',
level=logging.DEBUG)
import urllib2
rdata = {
"styleId": "chicago-author-date",
"citationGroups": [{
"citationItems": [{
"easyKey": 'martin:2012acts'
}],
"properties": {
"noteIndex": 0
}
}]
}
rdata = {
"citationGroups": [
{
"citationItems": [
{
"easyKey": "martin:2012acts",
"author-only": True
}
],
"properties": {
"index": 0,
"noteIndex": 0
}
},
{
"citationItems": [
{
"easyKey": "eusebius:1890eusebius",
"suppress-author": True
}
],
"properties": {
"index": 5,
"noteIndex": 0
}
}
],
"styleId": "chicago-author-date"
}
req = urllib2.Request('http://localhost:23119/zotxt/bibliography')
req.add_header('Content-Type', 'application/json')
handler = urllib2.HTTPHandler(debuglevel=2)
opener = urllib2.build_opener(handler)
res = opener.open(req, data=json.dumps(rdata))
logging.debug('res = getcode %s\n', res.getcode())
logging.debug('data:\n%s', res.read())
I get an error:
$ python play_bibliography.py
send: 'POST /zotxt/bibliography HTTP/1.1\r\nAccept-Encoding: identity\r\nContent-Length: 300\r\nHost: localhost:23119\r\nContent-Type: application/json\r\nConnection: close\r\nUser-Agent: Python-urllib/2.7\r\n\r\n{"citationGroups": [{"citationItems": [{"easyKey": "martin:2012acts", "author-only": true}], "properties": {"index": 0, "noteIndex": 0}}, {"citationItems": [{"easyKey": "eusebius:1890eusebius", "suppress-author": true}], "properties": {"index": 5, "noteIndex": 0}}], "styleId": "chicago-author-date"}'
reply: 'HTTP/1.0 400 Bad Request\r\n'
header: X-Zotero-Version: 4.0.28.1
header: X-Zotero-Connector-API-Version: 2
header: Content-Type: text/plain
Traceback (most recent call last):
File "play_bibliography.py", line 57, in <module>
res = opener.open(req, data=json.dumps(rdata))
File "/usr/lib64/python2.7/urllib2.py", line 437, in open
response = meth(req, response)
File "/usr/lib64/python2.7/urllib2.py", line 550, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib64/python2.7/urllib2.py", line 475, in error
return self._call_chain(*args)
File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/usr/lib64/python2.7/urllib2.py", line 558, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 400: Bad Request
$
By pchampin on 2015-09-11 15:23:34+00:00
I tested your script play_bibliography.py
, replacing the keys with some that exist in my Zotero, and it worked.
My guess is that some of the easyKeys in your data are invalid. I get the same error when I changed one of the keys for an invalid one. Note that Zotxt is really fussy about the words you use in the keys (it must be the exact same spelling), and if you have several references of the same author in the same year, you must be careful to chose a word in the title that exist only one of them (else the key is ambiguous). Finally, in my experience, sometimes Zotxt even rejects a key that seems perfectly valid and unambiguous; in those situations, picking a different word from the title solved the problem.
It is annoying that Zotxt fails with a 400 error (and no informative message) whenever a single key is incorrect in the query. I submitted an issue about that problem.
By mcepl on 2015-09-11 21:00:40+00:00
My guess is that some of the easyKeys in your data are invalid.
I agree with the issue. The proper error reporting is the key here (although I don't agree with 2## code as I have indicated in your issue).
By egh on 2015-09-12 05:01:35+00:00
Thanks so much, Pierre-Antoine & Matěj, and my apologies for the slowness of addressing this. I will try to fix zotxt to return a better error on a bad easykey.
By the way, if you prefer, you can use Better BibTeX keys instead. The workflow should be exactly the same - just use the Better BibTeX key instead.