Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

r.kappa: Fix failures, garbage output, fallback to category values #2573

Merged
merged 7 commits into from
Nov 3, 2022

Conversation

marisn
Copy link
Contributor

@marisn marisn commented Sep 9, 2022

r.kappa has some strange design decisions and produces strange or outright wrong output if input data doesn't match its expectations.
This PR fixes:

  • printing garbage instead of NA for the first raster category for commision, ommision and kappa values;
  • printing raster category values if labels are missing for matrix-only output mode;

Although original code lacks any explanation why NA should not be
printed for the first raster category, I do suspect it stems from
idea that the first cat is 0 and before proper NULL support 0
was "no data" value.
@marisn marisn added raster Related to raster data processing C Related code is in C labels Sep 9, 2022
@marisn marisn added this to the 8.3.0 milestone Sep 9, 2022
@marisn marisn changed the title R kappa fixes r.kappa fixes Sep 9, 2022
raster/r.kappa/prt2csv_mat.c Fixed Show resolved Hide resolved
raster/r.kappa/prt2csv_mat.c Fixed Show fixed Hide fixed
Copy link
Member

@wenzeslaus wenzeslaus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GCC and CodeQL are not happy. Otherwise no idea without testing or tests, sorry.

Probably another candidate for JSON output, but that's out of scope for this PR.

raster/r.kappa/prt2csv_mat.c Fixed Show resolved Hide resolved
@marisn
Copy link
Contributor Author

marisn commented Sep 14, 2022

GCC and CodeQL are not happy. Otherwise no idea without testing or tests, sorry.

There are more issues as some values are calculated incorrectly if input data is not sequential numbers starting at 0. Thus just a draft as more work is needed.

Probably another candidate for JSON output, but that's out of scope for this PR.

I was thinking of shell script style with "-g" flag, but going for JSON also is an option. In separate PR, of course, after this PR is merged as this PR will have to be backported.

These tests should fail as C code fixes will be in upcoming commits.
pii, pi, pj and pii all are arrays with size of ncat thus when
nstats < ncat it was causing out of bounds reads and writes.
A resulting segfault was reported in:
https://trac.osgeo.org/grass/ticket/3978

A test covering this error already was added in previous commit.
@marisn marisn marked this pull request as ready for review September 23, 2022 12:44
@marisn marisn requested a review from wenzeslaus September 23, 2022 12:44
Copy link
Member

@wenzeslaus wenzeslaus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code changes make sense to me in light of the individual commit messages (please include these descriptions into the final commit message, you could add pieces of them as comments too).

I executed the tests locally with the original code and in the one case where it actually runs and gives results, I get the same values as the ones in the tests, so the outputs which are working were not broken.

Thank you for the extra effort with the correctness test! I was really just thinking some "current values" check to see the before and after difference (or no difference).

I only noticed sub-optimal use of split in Python in tests which would be nice to have before merge.

raster/r.kappa/testsuite/test_r_kappa.py Outdated Show resolved Hide resolved
raster/r.kappa/testsuite/test_r_kappa.py Outdated Show resolved Hide resolved
@wenzeslaus wenzeslaus changed the title r.kappa fixes r.kappa: Fix failures, garbage output, fallback to category values Oct 5, 2022
@marisn marisn merged commit 8dd3ce7 into OSGeo:main Nov 3, 2022
neteler added a commit to neteler/grass that referenced this pull request Nov 10, 2022
Selected r.kappa indentation update from OSGeo#2544

(needed for backport r.kappa fixes in OSGeo#2573)
neteler added a commit that referenced this pull request Nov 10, 2022
Selected r.kappa indentation update from #2544

(needed for backport of r.kappa fixes in #2573)
neteler pushed a commit that referenced this pull request Nov 10, 2022
…2573)

* use raster category values in matrix output if labels are not present

* print NAs also for the first category
Although original code lacks any explanation why NA should not be
printed for the first raster category, I do suspect it stems from
idea that the first cat is 0 and before proper NULL support 0
was "no data" value.

* r.kappa: fix incorrect memory access and improve edge case handling

pii, pi, pj and pii all are arrays with size of ncat thus when
nstats < ncat it was causing out of bounds reads and writes.
A resulting segfault was reported in:
https://trac.osgeo.org/grass/ticket/3978

* tests for most of functionality
@neteler neteler modified the milestones: 8.3.0, 8.2.1 Nov 10, 2022
ninsbl pushed a commit to ninsbl/grass that referenced this pull request Feb 17, 2023
…SGeo#2573)

* use raster category values in matrix output if labels are not present

* print NAs also for the first category
Although original code lacks any explanation why NA should not be
printed for the first raster category, I do suspect it stems from
idea that the first cat is 0 and before proper NULL support 0
was "no data" value.

* r.kappa: fix incorrect memory access and improve edge case handling

pii, pi, pj and pii all are arrays with size of ncat thus when
nstats < ncat it was causing out of bounds reads and writes.
A resulting segfault was reported in:
https://trac.osgeo.org/grass/ticket/3978

* tests for most of functionality
marisn added a commit to marisn/grass that referenced this pull request Jun 2, 2023
…SGeo#2573)

* use raster category values in matrix output if labels are not present

* print NAs also for the first category
Although original code lacks any explanation why NA should not be
printed for the first raster category, I do suspect it stems from
idea that the first cat is 0 and before proper NULL support 0
was "no data" value.

* r.kappa: fix incorrect memory access and improve edge case handling

pii, pi, pj and pii all are arrays with size of ncat thus when
nstats < ncat it was causing out of bounds reads and writes.
A resulting segfault was reported in:
https://trac.osgeo.org/grass/ticket/3978

* tests for most of functionality
@marisn marisn deleted the r_kappa_fixes branch October 22, 2023 09:57
neteler pushed a commit to nilason/grass that referenced this pull request Nov 7, 2023
…SGeo#2573)

* use raster category values in matrix output if labels are not present

* print NAs also for the first category
Although original code lacks any explanation why NA should not be
printed for the first raster category, I do suspect it stems from
idea that the first cat is 0 and before proper NULL support 0
was "no data" value.

* r.kappa: fix incorrect memory access and improve edge case handling

pii, pi, pj and pii all are arrays with size of ncat thus when
nstats < ncat it was causing out of bounds reads and writes.
A resulting segfault was reported in:
https://trac.osgeo.org/grass/ticket/3978

* tests for most of functionality
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C Related code is in C raster Related to raster data processing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants