[v24.1.x] [CORE-7803] Audit Log Manager: Refactoring - use client::produce_record_batch and reduce retries.

[vbotbuildovich]

Backport #23775 to branch v24.1.x. Note that this issue was created as a placeholder, since the original PR's commit(s) could not be automatically cherry-picked.
Here are the commands to execute:

git checkout v24.1.x
git checkout -b vbotbuildovich/backport-23775-v24.1.x-221
git cherry-pick -x 090f41b02f 11b8a23255 37414a7d58 5c7ed51bff e5fd326728

git push origin vbotbuildovich/backport-23775-v24.1.x-221
gh pr create \
  --title "[v24.1.x] [CORE-7803] Audit Log Manager: Refactoring - use `client::produce_record_batch` and reduce retries." \
  --base "v24.1.x" \
  --label "kind/backport" \
  --head "vbotbuildovich/backport-23775-v24.1.x-221" \
  --draft \
  --repo "redpanda-data/redpanda" \
  --milestone "v24.1.x-next" \
  --body "Backport of PR /~https://github.com/redpanda-data/redpanda/pull/23775 "

[CORE-7803]: https://redpandadata.atlassian.net/browse/CORE-7803?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

[BenPope]

Fixed by #23868