secp256k1 is a test of whether this function has been implemented in another language or not

[NJdevPro]

If we want to compare language performance, the problems should be entirely implemented in the languages to compare.
Hence calling an implementation in another language should be ruled out because it tells us nothing about the languages tested.
For problem secp256k1, the Go implementation 0-ffi.go merely calls github.com/ethereum/go-ethereum/crypto/secp256k1 which itself calls C routines.

Not only it makes the speed comparison useless, but we can't compare the ease of implementation (number of lines of code) either.
All this tells us is that there exists an implementation of the secp256k1 algorithm out there. That information isn't very interesting per se in order to choose a language.

Granted, it also tells us that Go can interface with C, but that shouldn't be the purpose of a performance benchmark. That's just part of the language spec. So all this test is telling us is that there exists an optimized implementation of this particular algorithm out there; an algorithm that is particularly niche as it has no use at all besides cryptocurrencies, afaik (at least if it was an implementation of FFT, it would make more sense). And it's optimized because it calls C.
I would make an exception for the standard library that comes with the language, because we should consider that the standard library to be integral part of the language.
But for everything else, no.

One thing that could be tested is the speed and easiness of interfacing to C (or another language), but the test should focus on the exchange of data between the two languages (are there copies between C and host language ?), and not on how well C performs a computation.

[cyrusmsk]

Sone discussion could be found there: #220

author said go-ffi is exception from the rules