Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[LLDB] Make the thread list for SBSaveCoreOptions iterable #122541

Merged
merged 8 commits into from
Jan 16, 2025

Conversation

Jlalond
Copy link
Contributor

@Jlalond Jlalond commented Jan 10, 2025

This patch adds the ability to get a thread at a give index, based on insertion order, for SBSaveCore Options. This is primarily to benefit scripts using SBSaveCore, and remove the need to have both options and a second collection if your script is tracking what threads need to be saved. Such as if you want to collect the source of all the threads to be saved after the Core is generated.

@Jlalond Jlalond requested a review from bulbazord January 10, 2025 22:31
@Jlalond Jlalond requested a review from JDevlieghere as a code owner January 10, 2025 22:31
@llvmbot llvmbot added the lldb label Jan 10, 2025
@Jlalond Jlalond requested a review from clayborg January 10, 2025 22:32
@llvmbot
Copy link
Member

llvmbot commented Jan 10, 2025

@llvm/pr-subscribers-lldb

Author: Jacob Lalonde (Jlalond)

Changes

This patch adds the ability to get a thread at a give index, based on insertion order, for SBSaveCore Options. This is primarily to benefit scripts using SBSaveCore, and remove the need to have both options and a second collection if your script is tracking what threads need to be saved. Such as if you want to collect the source of all the threads to be saved after the Core is generated.


Full diff: /~https://github.com/llvm/llvm-project/pull/122541.diff

6 Files Affected:

  • (modified) lldb/include/lldb/API/SBSaveCoreOptions.h (+15)
  • (modified) lldb/include/lldb/Symbol/SaveCoreOptions.h (+6-2)
  • (modified) lldb/source/API/SBSaveCoreOptions.cpp (+13)
  • (modified) lldb/source/Symbol/SaveCoreOptions.cpp (+28-6)
  • (modified) lldb/test/API/python_api/sbsavecoreoptions/TestSBSaveCoreOptions.py (+42-3)
  • (modified) lldb/test/API/python_api/sbsavecoreoptions/basic_minidump.yaml (+10)
diff --git a/lldb/include/lldb/API/SBSaveCoreOptions.h b/lldb/include/lldb/API/SBSaveCoreOptions.h
index 74aa2fe5bd5f92..d480b6272749e6 100644
--- a/lldb/include/lldb/API/SBSaveCoreOptions.h
+++ b/lldb/include/lldb/API/SBSaveCoreOptions.h
@@ -111,6 +111,21 @@ class LLDB_API SBSaveCoreOptions {
   ///   style specific regions.
   SBError AddMemoryRegionToSave(const SBMemoryRegionInfo &region);
 
+  /// Get the number of Threads to be saved
+  ///
+  /// \returns
+  ///  The count of Threads to be saved.
+  uint32_t GetNumThreads() const;
+
+  /// Get the Thread at the specified index.
+  ///
+  /// \param [in] idx
+  ///   The index of the thread to return.
+  /// \returns
+  ///   The thread at the specified index, or an empty thread if the index is
+  ///   greater than or equal to the number of threads.
+  lldb::SBThread GetThreadAtIndex(uint32_t idx) const;
+
   /// Reset all options.
   void Clear();
 
diff --git a/lldb/include/lldb/Symbol/SaveCoreOptions.h b/lldb/include/lldb/Symbol/SaveCoreOptions.h
index d90d08026016dc..ebe68cb45c1216 100644
--- a/lldb/include/lldb/Symbol/SaveCoreOptions.h
+++ b/lldb/include/lldb/Symbol/SaveCoreOptions.h
@@ -13,7 +13,6 @@
 #include "lldb/Utility/RangeMap.h"
 
 #include <optional>
-#include <set>
 #include <string>
 #include <unordered_set>
 
@@ -47,6 +46,9 @@ class SaveCoreOptions {
 
   void AddMemoryRegionToSave(const lldb_private::MemoryRegionInfo &region);
 
+  std::optional<lldb::ThreadSP> GetThreadAtIndex(uint32_t idx) const;
+  uint32_t GetNumThreads() const;
+
   void Clear();
 
 private:
@@ -56,8 +58,10 @@ class SaveCoreOptions {
   std::optional<lldb_private::FileSpec> m_file;
   std::optional<lldb::SaveCoreStyle> m_style;
   lldb::ProcessSP m_process_sp;
-  std::unordered_set<lldb::tid_t> m_threads_to_save;
+  std::unordered_map<lldb::tid_t, lldb::ThreadSP> m_threads_to_save;
   MemoryRanges m_regions_to_save;
+
+  std::vector<lldb::tid_t> m_thread_indexes; // Indexes into m_threads_to_save
 };
 } // namespace lldb_private
 
diff --git a/lldb/source/API/SBSaveCoreOptions.cpp b/lldb/source/API/SBSaveCoreOptions.cpp
index c79b57fa62c2be..6e50145add822e 100644
--- a/lldb/source/API/SBSaveCoreOptions.cpp
+++ b/lldb/source/API/SBSaveCoreOptions.cpp
@@ -100,6 +100,19 @@ SBSaveCoreOptions::AddMemoryRegionToSave(const SBMemoryRegionInfo &region) {
   return SBError();
 }
 
+uint32_t lldb::SBSaveCoreOptions::GetNumThreads() const {
+  LLDB_INSTRUMENT_VA(this);
+  return m_opaque_up->GetNumThreads();
+}
+
+SBThread SBSaveCoreOptions::GetThreadAtIndex(uint32_t idx) const {
+  LLDB_INSTRUMENT_VA(this, idx);
+  std::optional<lldb::ThreadSP> thread_sp = m_opaque_up->GetThreadAtIndex(idx);
+  if (thread_sp)
+    return SBThread(thread_sp.value());
+  return SBThread();
+}
+
 void SBSaveCoreOptions::Clear() {
   LLDB_INSTRUMENT_VA(this);
   m_opaque_up->Clear();
diff --git a/lldb/source/Symbol/SaveCoreOptions.cpp b/lldb/source/Symbol/SaveCoreOptions.cpp
index 8d9aadece2152d..84c4a76f09453e 100644
--- a/lldb/source/Symbol/SaveCoreOptions.cpp
+++ b/lldb/source/Symbol/SaveCoreOptions.cpp
@@ -87,12 +87,33 @@ Status SaveCoreOptions::AddThread(lldb::ThreadSP thread_sp) {
     m_process_sp = thread_sp->GetProcess();
   }
 
-  m_threads_to_save.insert(thread_sp->GetID());
+  m_threads_to_save.insert({thread_sp->GetID(), thread_sp});
+  m_thread_indexes.push_back(thread_sp->GetID());
   return error;
 }
 
 bool SaveCoreOptions::RemoveThread(lldb::ThreadSP thread_sp) {
-  return thread_sp && m_threads_to_save.erase(thread_sp->GetID()) > 0;
+  if (!thread_sp)
+    return false;
+  if (m_threads_to_save.erase(thread_sp->GetID()) == 0)
+    return false;
+
+  auto it = std::find(m_thread_indexes.begin(), m_thread_indexes.end(),
+                      thread_sp->GetID());
+  m_thread_indexes.erase(it);
+  return true;
+}
+
+uint32_t SaveCoreOptions::GetNumThreads() const {
+  return m_threads_to_save.size();
+}
+
+std::optional<lldb::ThreadSP>
+SaveCoreOptions::GetThreadAtIndex(uint32_t idx) const {
+  if (idx >= m_thread_indexes.size())
+    return std::nullopt;
+  lldb::tid_t tid = m_thread_indexes[idx];
+  return m_threads_to_save.find(tid)->second;
 }
 
 bool SaveCoreOptions::ShouldThreadBeSaved(lldb::tid_t tid) const {
@@ -115,8 +136,8 @@ const MemoryRanges &SaveCoreOptions::GetCoreFileMemoryRanges() const {
   return m_regions_to_save;
 }
 
-Status SaveCoreOptions::EnsureValidConfiguration(
-    lldb::ProcessSP process_sp) const {
+Status
+SaveCoreOptions::EnsureValidConfiguration(lldb::ProcessSP process_sp) const {
   Status error;
   std::string error_str;
   if (!m_threads_to_save.empty() && GetStyle() == lldb::eSaveCoreFull)
@@ -132,10 +153,10 @@ Status SaveCoreOptions::EnsureValidConfiguration(
   return error;
 }
 
-void SaveCoreOptions::ClearProcessSpecificData() { 
+void SaveCoreOptions::ClearProcessSpecificData() {
   // Deliberately not following the formatter style here to indicate that
   // this method will be expanded in the future.
-  m_threads_to_save.clear(); 
+  m_threads_to_save.clear();
 }
 
 void SaveCoreOptions::Clear() {
@@ -145,4 +166,5 @@ void SaveCoreOptions::Clear() {
   m_threads_to_save.clear();
   m_process_sp.reset();
   m_regions_to_save.Clear();
+  m_thread_indexes.clear();
 }
diff --git a/lldb/test/API/python_api/sbsavecoreoptions/TestSBSaveCoreOptions.py b/lldb/test/API/python_api/sbsavecoreoptions/TestSBSaveCoreOptions.py
index 40d0cc7e96ff48..00fddbde345154 100644
--- a/lldb/test/API/python_api/sbsavecoreoptions/TestSBSaveCoreOptions.py
+++ b/lldb/test/API/python_api/sbsavecoreoptions/TestSBSaveCoreOptions.py
@@ -4,15 +4,18 @@
 from lldbsuite.test.decorators import *
 from lldbsuite.test.lldbtest import *
 
+
 class SBSaveCoreOptionsAPICase(TestBase):
     basic_minidump = "basic_minidump.yaml"
     basic_minidump_different_pid = "basic_minidump_different_pid.yaml"
 
     def get_process_from_yaml(self, yaml_file):
         minidump_path = self.getBuildArtifact(os.path.basename(yaml_file) + ".dmp")
-        print ("minidump_path: " + minidump_path)
+        print("minidump_path: " + minidump_path)
         self.yaml2obj(yaml_file, minidump_path)
-        self.assertTrue(os.path.exists(minidump_path), "yaml2obj did not emit a minidump file")
+        self.assertTrue(
+            os.path.exists(minidump_path), "yaml2obj did not emit a minidump file"
+        )
         target = self.dbg.CreateTarget(None)
         process = target.LoadCore(minidump_path)
         self.assertTrue(process.IsValid(), "Process is not valid")
@@ -59,7 +62,6 @@ def test_adding_and_removing_thread(self):
         removed_success = options.RemoveThread(thread)
         self.assertFalse(removed_success)
 
-
     def test_adding_thread_different_process(self):
         """Test adding and removing a thread from save core options."""
         options = lldb.SBSaveCoreOptions()
@@ -79,3 +81,40 @@ def test_adding_thread_different_process(self):
         self.assertTrue(error.Fail())
         error = options.AddThread(thread)
         self.assertTrue(error.Success())
+
+    def test_removing_and_adding_insertion_order(self):
+        """Test insertion order is maintained when removing and adding threads."""
+        options = lldb.SBSaveCoreOptions()
+        process = self.get_basic_process()
+        threads = []
+        for x in range(0, 3):
+            thread = process.GetThreadAtIndex(x)
+            threads.append(thread)
+            error = options.AddThread(thread)
+            self.assertTrue(error.Success())
+
+        # Get the middle thread, remove it, and insert it at the end.
+        middle_thread = threads[1]
+        self.assertTrue(options.RemoveThread(middle_thread))
+        num_threads = options.GetNumThreads()
+        self.assertEqual(num_threads, 2)
+        error = options.AddThread(middle_thread)
+        self.assertTrue(error.Success())
+        num_threads = options.GetNumThreads()
+        self.assertEqual(num_threads, 3)
+        thread_at_last_index = options.GetThreadAtIndex(2)
+        self.assertEqual(thread_at_last_index.id, middle_thread.id)
+        thread_at_middle_index = options.GetThreadAtIndex(1)
+        self.assertEqual(thread_at_middle_index.id, threads[2].id)
+
+        # Pop the front thread, remove it, and insert it at the end.
+        front_thread = threads[0]
+        self.assertTrue(options.RemoveThread(front_thread))
+        num_threads = options.GetNumThreads()
+        self.assertEqual(num_threads, 2)
+        error = options.AddThread(front_thread)
+        self.assertTrue(error.Success())
+        num_threads = options.GetNumThreads()
+        self.assertEqual(num_threads, 3)
+        thread_at_last_index = options.GetThreadAtIndex(2)
+        self.assertEqual(thread_at_last_index.id, front_thread.id)
diff --git a/lldb/test/API/python_api/sbsavecoreoptions/basic_minidump.yaml b/lldb/test/API/python_api/sbsavecoreoptions/basic_minidump.yaml
index 993c7da21225a9..96302fbfb6b5c2 100644
--- a/lldb/test/API/python_api/sbsavecoreoptions/basic_minidump.yaml
+++ b/lldb/test/API/python_api/sbsavecoreoptions/basic_minidump.yaml
@@ -24,3 +24,13 @@ Streams:
         Stack:
           Start of Memory Range: 0x00007FFFC8D0E000
           Content:               'DEADBEEF'
+      - Thread Id:       0x000074DE
+        Context:         0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000B0010000000000033000000000000000000000002020100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000040109600000000000100000000000000000000000000000068E7D0C8FF7F000068E7D0C8FF7F000097E6D0C8FF7F000010109600000000000000000000000000020000000000000088E4D0C8FF7F0000603FFF85C77F0000F00340000000000080E7D0C8FF7F000000000000000000000000000000000000E0034000000000007F0300000000000000000000000000000000000000000000801F0000FFFF00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF252525252525252525252525252525250000000000000000000000000000000000000000000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000FF00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
+        Stack:
+          Start of Memory Range: 0x00007FFFC8D0A000
+          Content:               'BEEFDEAD'
+      - Thread Id:       0x000074DF
+        Context:         0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000B0010000000000033000000000000000000000002020100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000040109600000000000100000000000000000000000000000068E7D0C8FF7F000068E7D0C8FF7F000097E6D0C8FF7F000010109600000000000000000000000000020000000000000088E4D0C8FF7F0000603FFF85C77F0000F00340000000000080E7D0C8FF7F000000000000000000000000000000000000E0034000000000007F0300000000000000000000000000000000000000000000801F0000FFFF00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF252525252525252525252525252525250000000000000000000000000000000000000000000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000FF00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
+        Stack:
+          Start of Memory Range: 0x00007FFFC8DFF000
+          Content:               'BAADBEEF'

/// \param [in] idx
/// The index of the thread to return.
/// \returns
/// The thread at the specified index, or an empty thread if the index is
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/an empty thread/an invalid SBThread object/

Not sure what users might think "an empty thread" is.

/// Get the Thread at the specified index.
///
/// \param [in] idx
/// The index of the thread to return.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/// The zero based index of the thread object to return.

@@ -47,6 +46,9 @@ class SaveCoreOptions {

void AddMemoryRegionToSave(const lldb_private::MemoryRegionInfo &region);

std::optional<lldb::ThreadSP> GetThreadAtIndex(uint32_t idx) const;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need optional if lldb::ThreadSP is what is returned, just return a lldb::ThreadSP and that can be checked for being invalid. There is no case where we want a valid optional with an empty lldb::ThreadSP.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can, I'm personally willing to trade the complexity for the performance wins. I think we have some minor complexity gain, primarily in the RemoveThread option, but the insertion case is quite simple. I rationally understand we may only today have a small number of threads, but I could imagine a world where we want to save a huge amount of threads, Potentially even GPU threads!

I agree the added complexity is ugly and only to maintain the insertion order. I think alternatively it would be nice if we could give out something akin to an SBIterator. As we only need to maintain the order today due to the caller only having access via index. Every usecase I can forsee will enumerate all threads, individual indexing isn't necessarily valuable.

Comment on lines 110 to 113
std::optional<lldb::ThreadSP> thread_sp = m_opaque_up->GetThreadAtIndex(idx);
if (thread_sp)
return SBThread(thread_sp.value());
return SBThread();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we remove the optional as suggested above, this can just be:

return SBThread(m_opaque_up->GetThreadAtIndex(idx));

@@ -56,8 +58,10 @@ class SaveCoreOptions {
std::optional<lldb_private::FileSpec> m_file;
std::optional<lldb::SaveCoreStyle> m_style;
lldb::ProcessSP m_process_sp;
std::unordered_set<lldb::tid_t> m_threads_to_save;
std::unordered_map<lldb::tid_t, lldb::ThreadSP> m_threads_to_save;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we need this complexity of an unordered_map + vector. Most users will add a couple of threads, or none at all. Can we just change this to be:

std::vector<lldb::tid_t> m_threads_to_save;

The get rid of m_thread_indexes?

Comment on lines 113 to 116
if (idx >= m_thread_indexes.size())
return std::nullopt;
lldb::tid_t tid = m_thread_indexes[idx];
return m_threads_to_save.find(tid)->second;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we only have a std::vector<tid_t> m_threads_to_save now, this can be:

if (!m_process_sp || idx >= m_threads_to_save.size())
  return ThreadSP();
return process_sp->GetThreadList().FindThreadByID(m_threads_to_save[idx], /*can_update=*/false);

Comment on lines 96 to 100
if (!thread_sp)
return false;
if (m_threads_to_save.erase(thread_sp->GetID()) == 0)
return false;

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we only have a std::vector<tid_t> m_threads_to_save now, these lines go away.

@Jlalond Jlalond force-pushed the sbsavecore-iteration branch from 194d6e3 to 7638844 Compare January 10, 2025 23:11
Copy link

github-actions bot commented Jan 11, 2025

✅ With the latest revision this PR passed the C/C++ code formatter.

@Jlalond Jlalond force-pushed the sbsavecore-iteration branch from d848e95 to a1c2d9d Compare January 11, 2025 00:17
@Jlalond
Copy link
Contributor Author

Jlalond commented Jan 11, 2025

@clayborg and I chatted offline. The need for indexable access wasn't as important as exposing an enumeration of threads. So we instead return a copy of the thread list in an SBThreadCollection.

@Jlalond Jlalond force-pushed the sbsavecore-iteration branch from 52cff74 to d6d45ad Compare January 13, 2025 17:27
@@ -79,3 +81,26 @@ def test_adding_thread_different_process(self):
self.assertTrue(error.Fail())
error = options.AddThread(thread)
self.assertTrue(error.Success())

def test_removing_and_adding_insertion_order(self):
"""Test insertion order is maintained when removing and adding threads."""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment for SBSaveCoreOptions::GetThreadsToSave says you get an unsorted collection back. Which one is the intended behavior?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey Alex, good catch. This shouldn't test the ordering, it did before when used our own internal collection but after Greg and I had some back and forth now we're returning a thread collection.

lldb/source/Symbol/SaveCoreOptions.cpp Outdated Show resolved Hide resolved
lldb/source/Symbol/SaveCoreOptions.cpp Outdated Show resolved Hide resolved
@Jlalond Jlalond merged commit 06edefa into llvm:main Jan 16, 2025
7 checks passed
DKLoehr pushed a commit to DKLoehr/llvm-project that referenced this pull request Jan 17, 2025
This patch adds the ability to get a thread at a give index, based on
insertion order, for SBSaveCore Options. This is primarily to benefit
scripts using SBSaveCore, and remove the need to have both options and a
second collection if your script is tracking what threads need to be
saved. Such as if you want to collect the source of all the threads to
be saved after the Core is generated.
Jlalond added a commit that referenced this pull request Jan 21, 2025
SBSaveCoreOptions has been around for awhile now, so I decided to draft
up some Docstrings describing the functionality better. Some of my
wording sounded a bit clunky due the optionality of each method call so
I would greatly appreciate feedback.

Includes the new method in #122541 so I will merge this as a follow up.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants