Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TASK] Use relative path to typo3lib in Apache Solr config #4310

Merged
merged 3 commits into from
Jan 30, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Docker/SolrServer/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ RUN rm -fR /opt/solr/server/solr/* \
&& groupmod --non-unique --gid "${SOLR_UNIX_GID}" solr \
&& chown -R solr:solr /var/solr /opt/solr \
&& apt update && apt upgrade -y && apt install sudo -y \
&& echo "solr ALL=NOPASSWD: /docker-entrypoint-initdb.d-as-sudo/*" > /etc/sudoers.d/solr
&& echo "solr ALL=NOPASSWD:SETENV: /docker-entrypoint-initdb.d-as-sudo/*" > /etc/sudoers.d/solr

COPY Docker/SolrServer/docker-entrypoint-initdb.d/ /docker-entrypoint-initdb.d
COPY Docker/SolrServer/docker-entrypoint-initdb.d-as-sudo/ /docker-entrypoint-initdb.d-as-sudo
Expand Down
Original file line number Diff line number Diff line change
@@ -1,28 +1,27 @@
#!/usr/bin/env bash

PATH_SOLRCONFIG="/var/solr/data/configsets/ext_solr_13_0_0/conf"
PATH_SOLRCONFIG="${SOLR_HOME}/configsets/ext_solr_13_0_0/conf"
PATH_AND_FILENAME_SOLRCONFIG="${PATH_SOLRCONFIG}/solrconfig.xml"

PATH_SOLRXML="/var/solr/data/"
PATH_AND_FILENAME_SOLRXML="${PATH_SOLRXML}/solr.xml"
PATH_AND_FILENAME_SOLRXML="${SOLR_HOME}/solr.xml"

if grep -q "<lib " "${PATH_AND_FILENAME_SOLRCONFIG}"; then
echo "The Apache Solr instance is affected on CVE-2025-24814"
echo " removing usages of <lib> tags in EXT:solr schemas and moving typo3lib from configset to solr.xml"

cp "${PATH_AND_FILENAME_SOLRCONFIG}" ${PATH_SOLRCONFIG}/solrconfig.xml.Backup-CVE-2025-24814
sed -i ':a;N;N;s/.*\<lib .*//g' ${PATH_AND_FILENAME_SOLRCONFIG}
cp "${PATH_AND_FILENAME_SOLRCONFIG}" "${PATH_SOLRCONFIG}/solrconfig.xml.Backup-CVE-2025-24814"
sed -i ':a;N;N;s/.*\<lib .*//g' "${PATH_AND_FILENAME_SOLRCONFIG}"

mv "${PATH_SOLRCONFIG}"/../typo3lib /var/solr/data/.
mv "${PATH_SOLRCONFIG}"/../typo3lib "${SOLR_HOME}/".

cp "${PATH_AND_FILENAME_SOLRXML}" ${PATH_SOLRXML}/solr.xml.Backup-CVE-2025-24814
cp "${PATH_AND_FILENAME_SOLRXML}" "${SOLR_HOME}/solr.xml.Backup-CVE-2025-24814"
# shellcheck disable=SC2016,SC1004
sed -i 's/<str name="modules">scripting<\/str>/<str name="modules">scripting,analytics,analysis-extras,langid,clustering,extraction,${solr.modules:}<\/str>\
<str name="allowPaths">${solr.allowPaths:}<\/str>\
<str name="allowUrls">${solr.allowUrls:}<\/str>\
\
<!-- TYPO3 Plugins -->\
<str name="sharedLib">\/var\/solr\/data\/typo3lib\/<\/str>/g' ${PATH_AND_FILENAME_SOLRXML}
<str name="sharedLib">typo3lib\/<\/str>/g' "${PATH_AND_FILENAME_SOLRXML}"
fi


Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ set -e
# execute files in /docker-entrypoint-initdb.d/as-sudo/*.sh before starting solr
while read -r f; do
case "$f" in
*.sh) echo "$0: running 'sudo $f'"; sudo "$f" ;;
*.sh) echo "$0: running 'sudo $f'"; sudo --preserve-env "$f" ;;
*) echo "$0: ignoring $f" ;;
esac
echo
Expand Down
61 changes: 37 additions & 24 deletions Documentation/GettingStarted/Solr.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ To pull the TYPO3 Solr image from Docker hub, simply type the following in comma

.. code-block:: bash

docker pull typo3solr/ext-solr:<EXT:Solr_Version>
docker pull typo3solr/ext-solr:<EXT:Solr_Version_main_version eg 13.0 or exact 13.0.1>

.. tip::

Expand All @@ -39,13 +39,13 @@ Our Docker image is based on `official Apache Solr image </~https://github.com/doc
This volume will be mounted to persist the index and other resources from Apache Solr server.
Following paths inside the exported volume are relevant for backups.

+---------------------------------------------------------------------------------------------------------+----------------------------------------------------------------+
| Path | Contents |
+=========================================================================================================+================================================================+
| data/data/<language> | the index data of corresponding core |
+---------------------------------------------------------------------------------------------------------+----------------------------------------------------------------+
| data/configsets/ext_solr_<EXT:Solr_Version>/conf/_schema_analysis_(stopwords\|synonyms)_<language>.json | the managed stop words and synonyms of corresponding core |
+---------------------------------------------------------------------------------------------------------+----------------------------------------------------------------+
+------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------+
| Path | Contents |
+========================================================================================================================+================================================================+
| data/data/<language> | the index data of corresponding core |
+------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------+
| data/configsets/ext_solr_<See_configset_on_version_matrix>/conf/_schema_analysis_(stopwords\|synonyms)_<language>.json | the managed stop words and synonyms of corresponding core |
+------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------+

.. tip::

Expand Down Expand Up @@ -95,7 +95,6 @@ Following commands will create the named volume "typo3s-solr-server-data" on hos

.. code-block:: yaml

version: '3.6'
services:
solr:
container_name: typo3s-solr-server
Expand Down Expand Up @@ -125,7 +124,10 @@ You should see the web interface of Solr to run queries:

.. figure:: /Images/GettingStarted/solr-query-webinterface.png

**Important**: The image ships a default cores for all languages. The data of the cores is stored on an exported volume. When you want to update the container, you can just start a new container using the data volume of the old container. But at the same time this has the limitation, that you should only use this image with the default cores! If you want to create custom cores with a different configuration please read the section "Advanced Docker Usage"
**Important**: The image ships a default cores for all languages. The data of the cores is stored on an exported volume.
When you want to update the container, you can just start a new container using the data volume of the old container.
But at the same time this has the limitation, that you should only use this image with the default cores!
If you want to create custom cores with a different configuration please read the section "Advanced Docker Usage"

Please note: The steps above show how to build the image from the Dockerfile. You can also download and use our compiled images from Docker Hub:

Expand All @@ -136,9 +138,11 @@ Advanced Docker Usage

Our image has the intension to create running cores out of the box. This implies, that the schema is inside the container.
The intension in our integration was to stay as close as possible to the official Apache Solr Docker images. Sometimes it might make
sence that you use the official image directly instead of our image. An example could be when you want to have the solrconfig, schema and data outside of the container.
sense that you use the official image directly instead of our image.
An example could be when you want to have the solrconfig, schema and data outside of the container.

The following example shows how you can run our configuration with the official Apache Solr Docker container by mounting the configuration and data from a volume (When using Docker on macOS make sure you've added the volume folder to "Preferences -> File Sharing").
The following example shows how you can run our configuration with the official Apache Solr Docker container by mounting the configuration
and data from a volume (When using Docker on macOS make sure you've added the volume folder to "Preferences -> File Sharing").

.. code-block:: bash

Expand All @@ -164,9 +168,9 @@ Known Installers

All of these installers can be used to setup a plain, reboot save Solr server:

* Use the installer shipped with Solr itself (bin/install_solr_service.sh):
* Use the installer shipped with Solr itself `bin/install_solr_service.sh`:

Allows to install Solr on many distributions including init scripts (At the time of development Ubuntu 16.04 was not supported and therefore it was no option for us to use it).
Allows to install Solr on many distributions including init scripts.

* Use chef / ansible / whatever dev ops tool:

Expand All @@ -177,28 +181,37 @@ e.g. https://galaxy.ansible.com/geerlingguy/solr/ (ansible) or https://supermark
Deployment of EXT:solr configuration into Apache Solr
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Since EXT:solr 6.0.0 the configuration and all JAR files are shipped in one "configSet". The goal of this approach is to make the deployment much easier.
Since EXT:solr 6.0.0 the configuration and all JAR files can be found in `EXT:solr/Resources/Private/Solr` folder.
The goal of this approach is to make the deployment much easier.

All you need to do is, you need to copy the configSet directory into your prepared Solr installation and replace the solr.xml file. In the installer we do it like this:

.. code-block:: bash

cp -r ${EXTENSION_ROOTPATH}/Resources/Private/Solr/configsets ${SOLR_INSTALL_DIR}/server/solr
cp ${EXTENSION_ROOTPATH}/Resources/Private/Solr/solr.xml ${SOLR_INSTALL_DIR}/server/solr/solr.xml
cp -r ${EXTENSION_ROOTPATH}/Resources/Private/Solr/* ${SOLR_INSTALL_DIR}/server/solr

After this, you can decide if you want to create the default cores by copying the default core.properties files or if you want to create a core with the Solr rest api.
After this, you can decide which cores you want to boot and delete not relevant cores from `${SOLR_INSTALL_DIR}/server/solr/cores/` directory.

Copy the default cores:
If you want to create other cores, you can create them by adding a `<core_name>/core.properties` file like provided by EXT:solr.
This is the recommended workflow, so you can deploy the Solr server in an Infrastructure as Code (IaC) manner.

.. code-block:: bash

cp -r ${EXTENSION_ROOTPATH}/Resources/Private/Solr/cores ${SOLR_INSTALL_DIR}/server/solr
Alternatively Create a core with the rest api **(not recommended)**:

Create a core with the rest api:
See: "V2 API" on https://solr.apache.org/guide/solr/latest/configuration-guide/coreadmin-api.html#coreadmin-create

.. code-block:: bash

curl "http://localhost:8983/solr/admin/cores?action=CREATE&name=core_de&configSet=ext_solr_8_0_0&schema=german/schema.xml&dataDir=../../data/german"
curl -X POST http://localhost:8983/api/cores -H 'Content-Type: application/json' -d '
{
"create": {
"name": "<core_name>",
"configSet": "ext_solr_<XX_Y_Z>",
"schema": "german/schema.xml",
"instanceDir": "cores/<core_name>"
"dataDir": "<absolute-path-to-SOLR-HOME-of_server(standard: /var/solr/data/)>/data/<core_name>"
}
}
'

After installing the Solr server and deploying all schemata, the TYPO3 reports module helps you to verify if your setup fits to the requirements of EXT:solr

Expand Down
4 changes: 2 additions & 2 deletions Documentation/Releases/solr-release-13-0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ No other steps are required.

Alternatively you can wipe the volume and start the container with v. 13.0.1+ image, but that method will wipe the index as well.

See the script `EXT:solr/Docker/SolrServer/docker-entrypoint-initdb.d/as-sudo/fix-CVE-2025-24814.sh`
See the script `EXT:solr/Docker/SolrServer/docker-entrypoint-initdb.d-as-sudo/fix-CVE-2025-24814.sh`


Other server setups
Expand Down Expand Up @@ -70,7 +70,7 @@ Steps:
<str name="allowUrls">${solr.allowUrls:}</str>

<!-- TYPO3 Plugins -->
<str name="sharedLib">/var/solr/data/typo3lib/</str>
<str name="sharedLib">typo3lib/</str>
#. Move the directory from `<Apache-Solr data dir>/configsets/ext_solr_13_0_0/typo3lib`

* to `<Apache-Solr data dir>/typo3lib`
Expand Down
2 changes: 1 addition & 1 deletion Resources/Private/Solr/solr.xml
Original file line number Diff line number Diff line change
Expand Up @@ -18,5 +18,5 @@
<str name="allowUrls">${solr.allowUrls:}</str>

<!-- TYPO3 Plugins -->
<str name="sharedLib">/var/solr/data/typo3lib/</str>
<str name="sharedLib">typo3lib/</str>
</solr>