find_ue function error in tutorial

[Milleupup]

i'm working on [Tutorial] ,
However,after i upgrade the package,this time find_ue return errors!!!

[jdlph]

Did you encounter a similar issue when you called find_shortest_path() in Section 3.1 prior to find_ue()?

[Milleupup]

no,it run successfully:shortest path (node id) from node 1 to node 2, distance: 3.06 mi | node path: 1;547;548;2.shortest path (link id) from node 1 to node 2, distance: 3.06 mi | link path: 1;986;989

[jdlph]

Thank you for the update! It helps verify that the C++ path engine works properly on your local machine. It seems that the root cause of your reported issue is also related to the path engine. Specifically, we suspect that the origin_node_no was passed to it as a null pointer.

As this is an isolated incident, please provide the following information for us to replicate this issue and conduct further investigation.

1. Input data.
2. A full screenshot includes the code snippet and error traces.
3. Your OS and build.
4. Python version.

You can directly attach data and images using this thread.

[Milleupup]

Thanks! The problem inexplicably disappeared, find_ue returns no errors(when i use my own dataset), but the output route_assignment file is empty?

[Milleupup]

Failed again...this time I use adviced dataset


python:3.11,path4gmns:0.9.9
OS:windows11
data:

[jdlph]

Would it be possible for you to zip your two data sets and attach them here so that we are able to replicate these two issues?

[Milleupup]

ok, there're my test dataset
Chicago_Sketch.zip

[jdlph]

Thank you for sharing your data sets! We are able to replicate these two issues.

The first issue (with respect to the empty route_assignment.csv) is caused by the data inconsistency on zone id representation between node.csv and demand.csv. Specifically, zone_id is decimal (e.g., "88.0" in Figure 1) in node.csv but integer in zone.csv (e.g., "1" in Figure 2) even though they are showing as integers in Excel. You can use an advanced text editor (e.g., Notepad++) to verify that. A similar issue can be found in #41.

Zone id is taken as string per GMNS specification and thus parsed as is. This data inconsistency results into no demand being loaded into the UE module. Therefore, you shall have seen zero gaps over the entire column updating process (as illustrated in Figure 3). Empty route_assignment.csv and zero volume for each link in link_performance.csv are expected.

After converting the decimal zone ids into integer values and changing proper units to read_network() (i.e., length_unit='meter', speed_unit='kph'), the issue is gone. Figures 4 and 5 show the terminal output and the first few rows in route_assignment.csv thereafter.

Figure 1 Zone id representation in node.csv

Figure 2 Zone id representation in demand.csv

Figure 3 Terminal output using the original MyData

Figure 4 Terminal output after converting decimal zone ids to integers

Figure 5 route_assignment.csv after converting decimal zone ids to integers

[jdlph]

The first issue regarding OSError is probably due to the execution of find_shortest_path() before find_ue() as shown in your code snippets. The issue was vanished after the invoking of find_shortest_path() is removed or placed after find_ue(). Please see the following screenshots for details.

The reason behind this issue is elaborated below.

find_ue() involves a preprocess of adding connectors and centroids to the underlying physical network (add_centroids_connectors()). However, the essential arrays for shortest path calculation are not updated accordingly per the current design of allocate_for_CAPI() in class Network. The column generation process then fails when it tries to find a shortest path tree as new nodes and links are never present in the following arrays.

1. from_node_no_arr
2. to_node_no_arr
3. first_links
4. last_links
5. sorted_link_no_arr

This happens to be a design flaw as we never thought of invoking find_shortest_path() and find_ue() in the same code snippet. This issue could be prevented from root if we had maintained a virtual network for find_ue() just like what we have done for class AccessNetwork.

Therefore, the simple and probably the only workaround is to remove the call(s) of find_shortest_path() prior to find_ue().

This issue will remain active until we come up with a permanent fix in the source code. Please feel free to add comments and suggestions using this thread.

[Milleupup]

thanks a lot, I'll check it later!

[jdlph]

v0.9.9.post1 is just released as a hotfix over the reported two issues.

Details regarding the first issue and how it was fixed can be found in the commit message of 2973475. Two new test cases are also added to unit testing to validate the fix. As it still indicates a design flaw for find_ue() under multiple agent types and multiple demand periods, we will leave this ticket open until we permanently fix it.

For the second one, the data inconsistency in terms of zone id representation shall be caught in the first place by the exception below.

Exception: volume is not encoded as numerical or Every zone id present in demand.csv is not found in node.csv.

For the latter one, different zone id representation could be the reason. For example, zone id is encoded as integer in demand.csv but decimal in node.csv. Note that zone_id CANNOT be decimal per GMNS specification!
Hint: use an advanced text editor (not Excel) to verify.