Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

missing eval script and links broken #12

Open
AnaValero opened this issue Oct 7, 2021 · 1 comment
Open

missing eval script and links broken #12

AnaValero opened this issue Oct 7, 2021 · 1 comment

Comments

@AnaValero
Copy link

Hi,

First of all, thanks for creating this promissing tool.
I've been following the github tutorial for DeepEventMine until I got the following error when running:
$ sh run.sh eval id gold dev sp
Error: "python: can't open file 'eval/scripts/eval-ev-id.py': [Errno 2] No such file or directory"
I found out that this folder only contains two scripts: eval-ev-cg.py and eval-ev-pc.py
Is it not uploaded yet or should I create it somehow?

Also, after running the steps "sh run.sh offset id gold test" and ".. gold dev", a message pops out that I don't really understand:
" submit this file: experiments/id/predict-gold-dev/ev-last/online-eval/id-dev-2021_10_06_18_25_54.tar.gz". (Same for id test)
I guess that these compressed files are to be sent through the links that appear in the github page: "ID Test", or "ID dev" n my case. However, these links are broken.

Could you bypass this latter step of submitting any created file to a webserver to enable a compact package for its fully ussage in house?

Thanks a lot"
I'm looking forward to use this tool with my data.
Ana

@AnaValero
Copy link
Author

AnaValero commented Oct 8, 2021

(Continuation)

Later, running it:
sh run.sh eval id gold dev p
I get:

  • Frequent error1:
    <_io.TextIOWrapper name='' mode='w' encoding='utf-8'> Failed to read experiments/id/predict-gold-dev/ev-last/ev-orig-a2/PMC1804205-00-TIAB.a2 : ERROR: duplicate ID E1 in experiments/id/predict-gold-dev/ev-last/ev-orig-a2/PMC1804205-00-TIAB.a2

  • Frequent error2:
    <_io.TextIOWrapper name='' mode='w' encoding='utf-8'> ##############################################################################
    PMID not in gold: PMC2651894-02-Results-03
    PMID not in annotation: PMC2651894-02-Results-03

  • Error3:
    <_io.TextIOWrapper name='' mode='w' encoding='utf-8'> # NOTE: SKIPPING experiments/id/predict-gold-dev/ev-last/ev-orig-a2/PMC1804205-02-Results-01.a2, READ FAILED: RESULTS WILL NOT BE VALID FOR THE FULL DATASET!
    <_io.TextIOWrapper name='' mode='w' encoding='utf-8'> ##############################################################################

  • Evaluation table for only 6/46 documents

I hope somebody had these issues and found out how to solve them...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant