-
Notifications
You must be signed in to change notification settings - Fork 224
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix calculation error when ref is empty #92
base: master
Are you sure you want to change the base?
Changes from 2 commits
1a74096
2877ab5
1114857
a62b637
2e22b51
3c235fe
0232a6c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -103,6 +103,36 @@ def test_compute_metrics_omit(self): | |
self.assertAlmostEqual(0.960771, scores['GreedyMatchingScore'], places=5) | ||
self.assertEqual(7, len(scores)) | ||
|
||
def test_compute_metricsa(self): | ||
n = NLGEval() | ||
|
||
# # Individual Metrics | ||
scores = n.compute_individual_metrics(ref=["Chocolate , a New Way to Get Cancer "], | ||
hyp="Chocolate , a Healthy Food") | ||
print("Chocolate , a New Way to Get Cancer ", scores) | ||
|
||
scores = n.compute_individual_metrics(ref=["Chocolate and an Interview "], | ||
hyp="Chocolate , a Healthy Food") | ||
print(scores) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can you assert something here, so that tests can detect regressions? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I found that this case is not necessary. I will remove it |
||
# | ||
# # Individual Metrics | ||
# scores = n.compute_individual_metrics(ref=["old people are helping in fact "], | ||
# hyp="old people are easy to get along with ") | ||
# print("old people are helping in fact ",scores) | ||
# | ||
# scores = n.compute_individual_metrics(ref=["old people are always happy "], | ||
# hyp="old people are easy to get along with ") | ||
# print("old people are always happy ",scores) | ||
|
||
# Individual Metrics | ||
# scores = n.compute_individual_metrics(ref=["help us to become a shy person"], | ||
# hyp="block our chances for a successful life") | ||
# print("help us to become a shy person", scores) | ||
# | ||
# scores = n.compute_individual_metrics(ref=["bring us great changes"], | ||
# hyp="block our chances for a successful life") | ||
# print(scores) | ||
|
||
def test_compute_metrics_empty(self): | ||
n = NLGEval() | ||
|
||
|
@@ -160,3 +190,11 @@ def test_compute_metrics(self): | |
self.assertAlmostEqual(0.568696, scores['VectorExtremaCosineSimilarity'], places=5) | ||
self.assertAlmostEqual(0.784205, scores['GreedyMatchingScore'], places=5) | ||
self.assertEqual(12, len(scores)) | ||
|
||
def test_compute_metricss(self): | ||
# The example from the README. | ||
root_dir = os.path.join(os.path.dirname(__file__), '..', '..') | ||
hypothesis = os.path.join(root_dir, 'examples/6.csv') | ||
references = os.path.join(root_dir, 'examples/7.pt_beam_1.csv') | ||
scores = nlgeval.compute_metrics(hypothesis, [references]) | ||
print(scores) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same, better to assert the result you expect. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I found that this case is not necessary. I will remove it |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you remove changes to your gitignore? They seem rather specific to your setup.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok I can remove it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
BTW You can also create a global gitignore: https://docs.github.com/en/github/getting-started-with-github/ignoring-files#configuring-ignored-files-for-all-repositories-on-your-computer
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@juharris Thank you for your advice. It is what I need.