Implement remaining builtins

[andychu]

This is a catch-all issue for people who are looking for small tasks.

There are various builtin commands that are unimplemented here, usually with a red X. (note: Oil language stuff is mostly not ready to tackle; contact me if you're not sure what that means.)

http://www.oilshell.org/release/0.6.pre16/doc/osh-quick-ref.html

It's best if you have a bit of experience using the builtin! I don't necessarily want to copy all of bash's builtins -- just the ones people use.

http://www.oilshell.org/release/0.5.alpha3/doc/osh-quick-ref.html

[andychu]

Example: We found that compgen -A functions and declare -F are similar and missing by running this script:

/~https://github.com/dylanaraps/pure-bash-bible/blob/master/test.sh

https://oilshell.zulipchat.com/#narrow/stream/121540-oil-discuss/topic/Tests

[jyn514]

I would like to see type show the definition of a function if possible.
osh:

cd is a function

bash:

$ type cd
cd is a function
cd () 
{ 
    if [ -z "$1" ]; then
        builtin cd && ls;
    else
        builtin cd "$1" && shift 2> /dev/null && ls "$@";
    fi
}

[jyn514]

I'm not sure how to work on this. I tracked it down to builtin_e in osh/builtin.py but it looks like that gets imported from _devbuild.gen.runtime_asdl and I don't know where that is or how to edit it.

Found this in osh/builtin.py:1010, def Type

[jyn514]

The builtin builtin is not supported, I can work on this. The other bit looks hard to implement.

[andychu]

OK great. I implemented command awhile ago, but I didn't see anyone using builtin. It looks like your function is using it, so OSH should have it.

The way I would tackle this is to copy your function into a case here:

/~https://github.com/oilshell/oil/blob/master/spec/builtins2.test.sh

Notes about spec tests:

/~https://github.com/oilshell/oil/wiki/Spec-Tests

Then run something like:

test/spec.sh builtins2 --verbose --range 6

And make sure the test is failing.

Then this change may give a clue (although I'm not entirely sure)

874d1f8

[jyn514]

Is there a way to resolve either a special or normal builtin? I saw Resolve and ResolveSpecial but those only work for one or the other, I want to allow both. The alternative is hacking around and doing BUILTIN_NAMES.get directly or modifying existing methods

I just called one after the other.

[jyn514]

Is there a way to tell the number of the currently executing line? All the major shells print <shell>: line <number>: builtin: <command>: not a shell builtin when there's an error in a script and I'd like to replicate that if possible

I found Mem.line_num but it's some custom string class I don't know how to use.

Mem.line_num.s works great.

[jyn514]

How do I find the name osh was invoked as? For example bash will say /bin/sh if invoked as sh.

Mem.GetArgNum(0) works, but it looks like this is always osh no matter how we were invoked.

[jyn514]

It looks like _devbuild/gen/runtime_asdl is autogenerated? I modified it locally but it's ignored by .gitignore, I'm not sure what source that corresponds to.

Tracked this down to somewhere in core/asdl_gen.py. I think this is something in osh/runtime.asdl actually, I'll mess with it later

[andychu]

Yes that file is generated from osh/runtime.asdl.

I wrote a few blog posts about ASDL, this one is an intro: http://www.oilshell.org/blog/2016/12/11.html

http://www.oilshell.org/blog/tags.html?tag=ASDL#ASDL

Why do you need to find the name osh is invoked as? That is sort of an ambiguous question in Unix, e.g. due to the presence of symlinks. There should be a way to do it, but it depends on what you need it for.

[jyn514]

It's not terribly important, I just thought it would be nice to be consistent with bash. If it's a pain, printing osh is still useful for the user.

[sethwoodworth]

I'd like to implement fc, which I use constantly in interactive mode. How do you suggest implementing it? I've started by creating a class in osh/builtins.py modeled after the class implementation of History. I see a Waiter class in core/process.py that sounds like it might fit the bill, and I could follow the existing path in osh/cmd_exec.py of creating a process, creating a tmpfile with the output of !!, which I can kind of understand from osh/history.py, executing an instance of the EDITOR variable with tmpfile, reading the process success after exit, reading from the tmpfile and executing it.
Any suggestions/pointers to get me started?

[andychu]

Hm good call, I honestly have never used fc, and now that I look at it, it seems useful! :-/ I have alias hi='history 30 and type the number. I guess with fc you still have to type the number? But it allows you to edit it, which is nice.

I always wanted some sort of interactive chooser ... but that's a different story.

---

Anyway I think you could just start it with subprocess.call() for the first pass? Does that make it easier? OSH doesn't "link" that library, but we can deal with it in a second pass.

What I mainly care about is testing. As long as code is tested, it's easy to refactor and minimize. fc is special because there is no other commands that invokes $EDITOR right now. So that makes it a little harder to test, but I think there is enough non-interactive stuff that can be tested. It looks like the -l flag provides a decent hook for testing logic apart from the details of launching the editor.

Following History seems like a good idea. It might be a good idea to create a new builtins_history.py file if that file is getting too long (although it's up to you.)

[xPMo]

Just mentioning that POSIX specifies command should implement -p, -v and -V flags. (Currently only the command some_prog form is supported) Ref: man 1p command

[andychu]

Yes I think there are Smoosh POSIX test suite covering that:

https://www.oilshell.org/release/0.8.pre2/test/spec.wwz/smoosh.html

Although my priority now is to increase the list of "real programs" run:

/~https://github.com/oilshell/oil/wiki/Shell-Programs-That-Run-Under-OSH

i.e. those are more important than features motivated only by POSIX. When people use Oil more, hopefully there will be some community effort to fix this long tail of issues. As I learned, it's a very long tail!

Of course I take patches to fix any POSIX compliance issues!

[andychu]

Also, as mentioned here, I will merge spec tests that show differences with existing shells, even if there isn't a patch yet. Specifying the behavior with tests goes a long way.

http://www.oilshell.org/blog/2020/03/recap.html#oil-08pre1

/~https://github.com/oilshell/oil/wiki/Spec-Tests

how to run spec tests: /~https://github.com/oilshell/oil/wiki/Contributing

And I do grep foo spec/*.test.sh to figure out what's already tested

[andychu]

• mapfile and readarray mentioned in disallowing redirects on control flow too strict by default (Run ble.sh) #620

[andychu]

Closing this since it's too general now; there are more specific bugs