I made an intellisense xml file for imgui.net

[jasonswearingen]

I'm new to ImGui and really wished there was intellisense explaining stuff, so I threw the generated cs file at Gemini and here's the result

https://gist.github.com/jasonswearingen/eaf56a82f9a4b03569edf4b9a48eb1c1

Please feel free to include this in the next nuget release. For those wanting to manually use it, paste it into the same dir as the nuget dll, overwrite existing. ex: C:\Users\jason\.nuget\packages\imgui.net\1.91.0.1\lib\net8.0

[sakitume]

This looks great! @jasonswearingen can you describe the steps you used to generate this .xml file? I do not understand "threw the generated cs file at Gemini" so a short description of the steps you took would be greatly appreciated.

[jasonswearingen]

honestly I don't really remember the steps. probably I downloaded this repo and built/run it, which much have generated a big .cs file with the CPP bindings. I would have then took that big file and pasted all it's text into gemini, and asked gemini to make an xml intellisense from it.

When I did this, gemini was the only LLM that supported big input sizes (iirc, 2million tokens) but it's output (was) still limited to 8000 tokens, so I had to tell it to "continue" a lot and copy/pasted the results of each run into a file, the result is the gist.

if you can find a LLM that supports large outputs, that would be the best for this. If you do, please mention it here (I haven't checked recently).

[BlueCyro]

Have you gone through and verified that the LLM didn't make any mistakes whatsoever? Generally they're good at simple data transformations like this, but you still need to make ABSOLUTELY sure that there are no missed or incorrect docs.

Anything done with an LLM should be under intense scrutiny for errors, and that scrutiny should be something you've done beforehand.

I'm more curious about why a script wasn't constructed to do this instead. That would have been a far more reliable way to generate docs, even if the script was something simple spat out by an LLM - as long as it was verified to work beforehand.

[jasonswearingen]

Have you gone through and verified that the LLM didn't make any mistakes whatsoever?

I only a very cursorily verification, and get benefit from the intellisense regardless of if it is not perfect. (It may be very great, or only okay. Having it is a lot, lot better than without IMO).

Generally they're good at simple data transformations like this, but you still need to make ABSOLUTELY sure that there are no missed or incorrect docs.

You are right that LLM's are imperfect, but I disagree that, in this circumstance, I need to 100% validate the generated intellisense. It's "just" a hint to help figure out which api to use, so IMO, imprecision is acceptable.

I'm more curious about why a script wasn't constructed to do this instead.

The main issue with this is that there are actually no real API docs for Imgui (that I can find). Secondly, assuming docs could be found, it's at least a few days of work to write said script. It took me a few hours to generate the attached intellisense, start-to-finish.

[BlueCyro]

I only a very cursorily verification, and get benefit from the intellisense regardless of if it is not perfect. (It may be very great, or only okay. Having it is a lot, lot better than without IMO).

You are right that LLM's are imperfect, but I disagree that, in this circumstance, I need to 100% validate the generated intellisense. It's "just" a hint to help figure out which api to use, so IMO, imprecision is acceptable.

I quite highly disagree that having errors in your intellisense is acceptable - especially when you consider that there are beginners using ImGui all the time. Mistakes in wording or incorrectly-documented functions can leave people pulling their hair out for hours as they find seemingly conflicting information. Intellisense should give you correct hints, not kind-of-correct-but-maybe-not-really ones.

The main issue with this is that there are actually no real API docs for Imgui (that I can find). Secondly, assuming docs could be found, it's at least a few days of work to write said script. It took me a few hours to generate the attached intellisense, start-to-finish.

There are. All over the place in fact. If you were to look at the imgui source, you can see inline documentation out the wazoo. It's not 100% coverage, but it's more than enough. You can see me hovering the definition of a struct and it displaying the contents of the comment peeking out behind it:

Many fields and functions have these comments that will show up in intellisense if you use the library over in C++ land. A slightly-less-than-complicated script could scrape all of these with 100% accuracy; unlike an LLM.

It doesn't matter if the LLM gets it almost right, because anything short of 100% right is, IMO, unfit for integration. I obviously can't speak for the maintainers of this library, but I would be colored quite the shade of surprised if unverified docs were accepted. Sorry, just gonna be blunt with that.

We should use tools required for the task at hand, and I really don't think that most LLMs are suited for tasks where accuracy is a concern - which is a category that documentation certainly fits into.

[jasonswearingen]

I agree that it would be fantastic to have intellisense.xml generated from the code/comments. Given that no such tool exists, we can agree to disagree that a LLM generated intellisense has value. 🤷

I'll leave this open for the project moderator/owner to close, just in the off-chance they find the linked intellisense useful.