Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve parse method in neo-cli #3204

Merged
merged 30 commits into from
May 20, 2024
Merged

Conversation

chenzhitong
Copy link
Member

@chenzhitong chenzhitong commented Apr 24, 2024

close #3201

I didn't add the Mnemonic conversion because it would require bringing in other NuGet packages. For Mnemonic conversion you can use the web version of the conversion tool https://neo.org/converter/index

@chenzhitong
Copy link
Member Author

@Jim8y @shargon please review

/// <returns>Returns null when the string does not represent a private key; otherwise
/// returns the string that represents the converted public key.
/// </returns>
public static string? PrivateKeyToPublicKey(string wif)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wif to pubkey?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wif to pubkey?

Yeah. What's the problem?

Copy link
Contributor

@Jim8y Jim8y Apr 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nothing,just thinking maybe you can make it both support wif and privatkey @shargon

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It worries me that people use a private key so openly, why not create a wallet?

//Initialize all OpCodes
var OperandSizePrefixTable = new int[256];
var OperandSizeTable = new int[256];
foreach (FieldInfo field in typeof(OpCode).GetFields(BindingFlags.Public | BindingFlags.Static))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use Script and GetInstruction?

Copy link
Member

@cschuchardt88 cschuchardt88 Apr 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes use Script class or something like this

byte[] data = [];

foreach (var d in data)
{
    Console.WriteLine(Enum.GetName((OpCode)d));
}

Comment on lines 407 to 419
private string? AddressToScripthashLE(string address)
{
try
{
var bigEndScript = address.ToScriptHash(NeoSystem.Settings.AddressVersion);

return bigEndScript.ToArray().ToHexString();
}
catch
{
return null;
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please add the following compatibility test for this API: convert address NRQJbjKWgcLX214Kbx6PDH2CHJdqbsSi4Q to LE script hash. I'd like to ensure that NeoGo has the same LE/BE conversion result:

anna@kiwi:~/Documents/GitProjects/nspcc-dev/neo-go$ ./bin/neo-go util convert NRQJbjKWgcLX214Kbx6PDH2CHJdqbsSi4Q
Address to BE ScriptHash	3c347b368199840085f7d92704a14becd578fa3d
Address to LE ScriptHash	3dfa78d5ec4ba10427d9f78500849981367b343c
Address to Base64 (BE)		PDR7NoGZhACF99knBKFL7NV4+j0=
Address to Base64 (LE)		Pfp41exLoQQn2feFAISZgTZ7NDw=
String to Hex				4e52514a626a4b5767634c583231344b6278365044483243484a6471627353693451
String to Base64			TlJRSmJqS1dnY0xYMjE0S2J4NlBESDJDSEpkcWJzU2k0UQ==

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

public static UInt160? TryParseUInt160(string? value)
{
    if (string.IsNullOrEmpty(value)) return default;
    if (UInt160.TryParse(value, out var result))
        return result;
    return UInt160.Zero;
}

public string? BigEndianToLittleEndian(string hex)
{
hex = hex.ToLower();
if (!new Regex("^0x([0-9a-f]{2})+$").IsMatch(hex)) return null;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make 0x prefix optional?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just check and remove it

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already modified

Comment on lines 745 to 756
if (op.ToString().StartsWith("PUSHINT"))
{
result.Add($"{op} {new BigInteger(operand)}");
}
else if (op == OpCode.SYSCALL)
{
result.Add($"{op} {dic[BitConverter.ToUInt32(operand)]}");
}
else
{
result.Add($"{op} {operand.ToHexString()}");
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

@cschuchardt88 cschuchardt88 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Dont use RegEx is so expensive and cost way to much CPU and can hang forever.
  2. Your naming convention. Change all methods to the proper names. To many for me to change for you; in a review. If you want I can change for you in Visual Studio; just would need access to your repo.

var parseFunctions = new Dictionary<string, Func<string, string?>>()
{
{ "Address to ScriptHash", AddressToScripthash },
{ "Address to ScriptHash (big-endian)", AddressToScripthash },
Copy link
Member

@cschuchardt88 cschuchardt88 Apr 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use Attribute class for the function description?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, I'll do it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already modified

/// </returns>
private string? NumberToHexToString(string input)
{
if (!new Regex("^\\d+$").IsMatch(input) || input.StartsWith('0'))
Copy link
Member

@cschuchardt88 cschuchardt88 Apr 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TryCatch is faster and cheaper for checking HexString encoding. Just TryCatch Convert.ToHexString and look for FormatException

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already modified

/// <returns>Correct Base64 string</returns>
public static string Base64Fixed(string str)
{
MatchCollection mc = Regex.Matches(str, @"\\u([\w]{2})([\w]{2})", RegexOptions.Compiled | RegexOptions.IgnoreCase);
Copy link
Member

@cschuchardt88 cschuchardt88 Apr 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TryCatch is faster and cheaper for checking Base64 encoding. Just TryCatch Convert.FromBase64String and look for FormatException

result.Add($"{op}");
}
}
return "\r\n" + string.Join("\r\n", result.ToArray());
Copy link
Member

@cschuchardt88 cschuchardt88 Apr 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use Environment.NewLine we do support different Operating Systems.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already modified

@chenzhitong
Copy link
Member Author

  1. Dont use RegEx is so expensive and cost way to much CPU and can hang forever.
  2. Your naming convention. Change all methods to the proper names. To many for me to change for you; in a review. If you want I can change for you in Visual Studio; just would need access to your repo.

I invited you. Thanks.
/~https://github.com/chenzhitong/neo/invitations

private string? BigEndianToLittleEndian(string hex)
{
bool hasHexPrefix = hex.StartsWith("0x", StringComparison.InvariantCultureIgnoreCase);
hex = hex[2..];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will throw an exception if the input is "a"


namespace Neo.CLI
{
public class ParseFunctionAttribute : Attribute
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One file per class

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless it has Interface with it.

[ParseFunction("Public Key to Address")]
private string? PublicKeyToAddress(string pubKey)
{
bool IsValidPubKey(string pubKey)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move method outside

Comment on lines 524 to 537
if (pubKey.Length != 66) return false;
if (pubKey[0] != '0') return false;
if (pubKey[1] != '2' && pubKey[1] != '3') return false;

for (var i = 2; i < pubKey.Length; i++)
{
var c = pubKey[i];
if (!((c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F')))
{
return false;
}
}

return true;
Copy link
Member

@cschuchardt88 cschuchardt88 May 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (pubKey.Length != 66) return false;
if (pubKey[0] != '0') return false;
if (pubKey[1] != '2' && pubKey[1] != '3') return false;
for (var i = 2; i < pubKey.Length; i++)
{
var c = pubKey[i];
if (!((c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F')))
{
return false;
}
}
return true;
return ECPoint.TryParse(pubkey, ECCurve.Secp256r1, out _);

Copy link
Member

@cschuchardt88 cschuchardt88 May 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

public keys can be many lengths and prefixes. Look at Neo.Cryptography.ECC.ECPoint class in method TryParse, Parse, DecodePoint or FromBytes

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already removed IsValidPubKey method

Copy link
Member

@cschuchardt88 cschuchardt88 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove all the hex methods in the file. No need for it. DotNet has Convert.ToHexString or Convert.FromHexString built-in.

/// string or when the converted string is not printable; otherwise, returns
/// the string represented by the hexadecimal value
/// </returns>
[ParseFunction("Hex String to String")]
private string? HexToString(string hexString)
{
try
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be replaced with Convert.ToHexString

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already removed HexToString StringToHex NumberToHexToString HexToNumber Base64ToHexString HexScriptsToOpCode method

Comment on lines 101 to 102
if (!IsHex(hex)) return null;
return "0x" + hex.HexToBytes().Reverse().ToArray().ToHexString(); ;
Copy link
Member

@cschuchardt88 cschuchardt88 May 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (!IsHex(hex)) return null;
return "0x" + hex.HexToBytes().Reverse().ToArray().ToHexString(); ;
Convert.FromHexString(hex); // check for is Hex
var bytes = System.Text.Encoding.UTF8.GetBytes(hex);
return "0x" + Convert.ToHexString([.. bytes.Reverse()]).ToLower();

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will output an uppercase string

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There I changed to LowerCase.

Comment on lines 118 to 121
bool hasHexPrefix = hex.StartsWith("0x", StringComparison.InvariantCultureIgnoreCase);
hex = hasHexPrefix ? hex[2..] : hex;
if (!hasHexPrefix || !IsHex(hex)) return null;
return hex.HexToBytes().Reverse().ToArray().ToHexString();
Copy link
Member

@cschuchardt88 cschuchardt88 May 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
bool hasHexPrefix = hex.StartsWith("0x", StringComparison.InvariantCultureIgnoreCase);
hex = hasHexPrefix ? hex[2..] : hex;
if (!hasHexPrefix || !IsHex(hex)) return null;
return hex.HexToBytes().Reverse().ToArray().ToHexString();
Convert.FromHexString(hex); // Check for hex
var bytes = System.Text.Encoding.UTF8.GetBytes(hex);
return Convert.ToHexString([.. bytes.Reverse()]);

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will crash when parse "0x3ff68d232a60f23a5805b8c40f7e61747f6f61ce"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrap in TryCatch

/// </returns>
/// <param name="str">Base64 strings containing unicode</param>
/// <returns>Correct Base64 string</returns>
public string Base64Fixed(string str)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe Console.OutputEncoding = Utility.StrictUTF8; will fix this problem.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The purpose of this one is to handle the output of other wallets or SDKs that have Base64 with Unicode escaped characters in them

Comment on lines 365 to 367
var bigEndScript = address.ToScriptHash(NeoSystem.Settings.AddressVersion);

return bigEndScript.ToArray().ToHexString();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
var bigEndScript = address.ToScriptHash(NeoSystem.Settings.AddressVersion);
return bigEndScript.ToArray().ToHexString();
return address.ToScriptHash(NeoSystem.Settings.AddressVersion);

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will output a result of type UInt160, not a string.

Comment on lines 513 to 514
var bytes = Convert.FromBase64String(base64.Trim());
return bytes.ToHexString();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
var bytes = Convert.FromBase64String(base64.Trim());
return bytes.ToHexString();
var bytes = Convert.FromBase64String(base64.Trim());
return $"0x{Convert.ToHexString(bytes)}";

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already modified

Comment on lines 548 to 549
if (!IsValidPubKey(pubKey)) return null;
return Contract.CreateSignatureContract(ECPoint.Parse(pubKey, ECCurve.Secp256r1)).ScriptHash.ToAddress(NeoSystem.Settings.AddressVersion);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (!IsValidPubKey(pubKey)) return null;
return Contract.CreateSignatureContract(ECPoint.Parse(pubKey, ECCurve.Secp256r1)).ScriptHash.ToAddress(NeoSystem.Settings.AddressVersion);
if (ECPoint.TryParse(pubKey, ECCurve.Secp256r1, out var publicKey) == false)
return null;
return Contract.CreateSignatureContract(publicKey)
.ScriptHash
.ToAddress(NeoSystem.Settings.AddressVersion);

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already modified

{
var privateKey = Wallet.GetPrivateKeyFromWIF(wif);
var account = new KeyPair(privateKey);
return account.PublicKey.ToArray().ToHexString();
Copy link
Member

@cschuchardt88 cschuchardt88 May 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return account.PublicKey.ToArray().ToHexString();
return Convert.ToHexString(account.PublicKey.EncodePoint(true)).ToLower();

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will output an uppercase string

Comment on lines 578 to 579
var pubKey = WIFToPublicKey(wif);
return Contract.CreateSignatureContract(ECPoint.Parse(pubKey, ECCurve.Secp256r1)).ScriptHash.ToAddress(0x35);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
var pubKey = WIFToPublicKey(wif);
return Contract.CreateSignatureContract(ECPoint.Parse(pubKey, ECCurve.Secp256r1)).ScriptHash.ToAddress(0x35);
var privateKey = Wallet.GetPrivateKeyFromWIF(wif);
var account = new KeyPair(privateKey);
return account.PublicKeyHash.ToAddress(NeoSystem.Settings.AddressVersion);

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two results are not the same

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh sorry, but what's the 0x35 for?

@chenzhitong
Copy link
Member Author

Can this be merged?

Copy link
Member

@cschuchardt88 cschuchardt88 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also please run dotnet format we have made changes the .editorconfig to re-run .github action tests.

Comment on lines 148 to 149
private bool IsHex(string str) => str.Length % 2 == 0 && str.All(c => (c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F'));

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will always be false because of the prefix 0x having the x in it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is by design, only for hexstring without '0x'

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dotnet format passed with no changes

cschuchardt88
cschuchardt88 previously approved these changes May 10, 2024
Copy link
Member

@cschuchardt88 cschuchardt88 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😄

@chenzhitong
Copy link
Member Author

Added ToLower() for LowerCase hexes.

Already modified, and ToHexString() is already lowercase.

@chenzhitong
Copy link
Member Author

chenzhitong commented May 11, 2024

Can this be merged? @shargon @cschuchardt88 @Jim8y

@cschuchardt88
Copy link
Member

Need a minimum of two people to approve on a PR before you can merge it.

Jim8y
Jim8y previously approved these changes May 12, 2024
shargon
shargon previously approved these changes May 13, 2024
Copy link
Member

@vncoelho vncoelho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but I need one week to test.
We are still investigating current release and adjusting our tools.

@Jim8y
Copy link
Contributor

Jim8y commented May 14, 2024

Let us wait until the 3.7 mainnet is online please.

@vncoelho
Copy link
Member

I think that we should wait a little as @Jim8y emphasized.

@shargon shargon added Blocked This issue can't be worked at the moment Cosmetic Type - Changes that improve user experience without affecting current functionality. labels May 16, 2024
@shargon shargon merged commit 8b687b6 into neo-project:master May 20, 2024
6 checks passed
Jim8y added a commit to Jim8y/neo that referenced this pull request May 25, 2024
…gins

* 'latest-plugins' of github.com:Jim8y/neo: (21 commits)
  fix: custom plugins won't shown by command `plugins` (neo-project#3269)
  COVERALL: Improve maintenance and readbility of some variables (neo-project#3248)
  Update nuget (neo-project#3262)
  [**Part-2**] Neo module/master fixes (neo-project#3244)
  Fix `dotnet pack` error (neo-project#3266)
  Fix and Update devcontainer.json to use Dockerfile  (neo-project#3259)
  Add optimization to template (neo-project#3247)
  Optimize plugin's models (neo-project#3246)
  fix CancelTransaction !signers.Any() (neo-project#3263)
  COVERALL: fix broken by changing report from lcov to cobertura (neo-project#3252)
  fix TraverseIterator count (neo-project#3261)
  Native: include DeprecatedIn hardfork into usedHardforks (neo-project#3245)
  [**Part-1**] `neo-module/master` (neo-project#3232)
  Make `ApplicationEngine.LoadContext` protection level `public` (neo-project#3243)
  improve parse method in neo-cli (neo-project#3204)
  Fix neo-project#3239 (neo-project#3242)
  Neo.CLI: enable hardforks for NeoFS mainnet (neo-project#3240)
  v3.7.4 (neo-project#3237)
  fix hardfork issues (neo-project#3234)
  Update src/Neo.CLI/CLI/MainService.Plugins.cs
  ...

# Conflicts:
#	src/Neo.CLI/CLI/MainService.Plugins.cs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Blocked This issue can't be worked at the moment Cosmetic Type - Changes that improve user experience without affecting current functionality.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

The parse method needs to be improved
6 participants