Remove unlimited collections from the repository API (eclipse-hawkbit…

…#496) * Started to get rid of unlimited collections Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Align API usage. Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * fix compile issues. Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Fix tests. Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Remove comments Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Performance optimizations. Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Remove dead code. Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Allign method names Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Wait until the action update event is processed Conflicts: hawkbit-dmf-amqp/src/test/java/org/eclipse/hawkbit/integration/AmqpMessageHandlerServiceIntegrationTest.java Signed-off-by: SirWayne <dennis.melzer@bosch-si.com> * Started new tag APIs Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Quotas into central interface. Tag tests added. Event names fixed. Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Simplified consumer run for every tenant. Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * remove unused fields. Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Alligned beans. Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Deprecated client methods for old resources. Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Fix new foreach method. Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Fix transaction for foreach. Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Extended DS creating to handle larger volumes. Fix on Readme. Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Fixed simulator bug and cleaned up tests. Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Fix in sorting. Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Remove configuration processor. Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Fix wrong usage of sanitize. Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Missing brackets. Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Fix README API compatability. Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Fix misinterpretation of pessimistic locking exceptions. Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Fix stability sentence. Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Code cleanup. Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com> * Fixed page calculation Signed-off-by: kaizimmerm <kai.zimmermann@bosch-si.com>
bosch-io · May 9, 2017 · a4e829e · a4e829e
1 parent cf7bdb8
commit a4e829e
Show file tree

Hide file tree

Showing 199 changed files with 3,500 additions and 1,605 deletions.
diff --git a/README.md b/README.md
@@ -100,4 +100,7 @@ $ java -jar ./examples/hawkbit-example-mgmt-simulator/target/hawkbit-example-mgm
 
 hawkBit is currently in '0.X' semantic version. That is due to the need that there is still content in hawkBit that is in need for refactoring. That includes the maven module structure, Spring Boot Properties, Spring Boot auto configuration as well as internal Java APIs (e.g. the [repository API](/~https://github.com/eclipse/hawkbit/issues/197) ).
 
-However, the external APIs (i.e. [Management API](/~https://github.com/eclipse/hawkbit/tree/master/hawkbit-mgmt-api), [DDI API](/~https://github.com/eclipse/hawkbit/tree/master/hawkbit-ddi-api), [DDI Artifact Download API](/~https://github.com/eclipse/hawkbit/tree/master/hawkbit-ddi-dl-api) and [DMF API](/~https://github.com/eclipse/hawkbit/tree/master/hawkbit-dmf/hawkbit-dmf-api) are on major version 'v1' and will be kept stable.
+However, the device facing [DDI API](/~https://github.com/eclipse/hawkbit/tree/master/hawkbit-ddi-api) is on major version 'v1' and will be kept stable.
+
+Server facing and [DMF API](/~https://github.com/eclipse/hawkbit/tree/master/hawkbit-dmf/hawkbit-dmf-api) are [Management API](/~https://github.com/eclipse/hawkbit/tree/master/hawkbit-mgmt-api) are on v1 as well. However, we cannot fully guarantee the same stability during hawkBit's 0.X development but we will try as best we can.
+
diff --git a/examples/hawkbit-device-simulator/README.md b/examples/hawkbit-device-simulator/README.md
@@ -27,8 +27,10 @@ This can be configured/disabled by spring boot properties
 ## hawkBit APIs
 
 The simulator supports `DDI` as well as the `DMF` integration APIs.
-In case there is no AMQP message broker (like rabbitMQ) running, you can disable the AMQP support for the device simulator, so the simulator is not trying to connect to an amqp message broker. 
-Configuration property `hawkbit.device.simulator.amqp.enabled=true` 
+
+In case there is no AMQP message broker (like rabbitMQ) running, you can disable the AMQP support for the device simulator, so the simulator is not trying to connect to an amqp message broker.
+
+Configuration property `hawkbit.device.simulator.amqp.enabled=false`
 
 ## Usage
 

diff --git a/...mulator/src/main/java/org/eclipse/hawkbit/mgmt/client/scenarios/ConfigurableScenario.java b/...mulator/src/main/java/org/eclipse/hawkbit/mgmt/client/scenarios/ConfigurableScenario.java
@@ -8,6 +8,7 @@
  */
 package org.eclipse.hawkbit.mgmt.client.scenarios;
 
+import java.math.BigDecimal;
 import java.security.SecureRandom;
 import java.util.List;
 import java.util.Random;
@@ -316,31 +317,38 @@ private void waitUntilRolloutIsReady(final Long id) {
 
     private void createDistributionSets(final Scenario scenario) {
         LOGGER.info("Creating {} distribution sets", scenario.getDistributionSets());
+        final BigDecimal pages = new BigDecimal(scenario.getDistributionSets())
+                .divide(new BigDecimal(PAGE_SIZE), BigDecimal.ROUND_UP).max(new BigDecimal(1));
+
+        IntStream.range(0, pages.intValue()).parallel().forEach(i -> createDistributionSetPage(scenario, i));
+        LOGGER.info("Creating {} distribution sets -> Done", scenario.getDistributionSets());
+    }
+
+    private void createDistributionSetPage(final Scenario scenario, final int page) {
 
         final List<MgmtDistributionSet> sets = distributionSetResource
                 .createDistributionSets(new DistributionSetBuilder().name(scenario.getDsName()).type("os_app")
-                        .version("1.0.").buildAsList(scenario.getDistributionSets()))
+                        .version("1.0.").buildAsList(calculateOffset(page),
+                                (page + 1) * PAGE_SIZE > scenario.getDistributionSets()
+                                        ? (scenario.getDistributionSets() - calculateOffset(page)) : PAGE_SIZE))
                 .getBody();
 
         assignSoftwareModulesTo(scenario, sets);
 
-        tagDistributionSets(scenario, sets);
+        tagDistributionSets(page, sets);
 
-        LOGGER.info("Creating {} distribution sets -> Done", scenario.getDistributionSets());
     }
 
-    private void tagDistributionSets(final Scenario scenario, final List<MgmtDistributionSet> sets) {
-        for (int i = 0; i < scenario.getDsTags(); i++) {
-            final MgmtTag tag = distributionSetTagResource
-                    .createDistributionSetTags(
-                            new TagBuilder().name("DS Tag" + i).description("DS tag for DS " + i).build())
-                    .getBody().get(0);
+    private void tagDistributionSets(final int page, final List<MgmtDistributionSet> sets) {
+        final MgmtTag tag = distributionSetTagResource
+                .createDistributionSetTags(
+                        new TagBuilder().name("Page " + page).description("DS tag for DS page" + page).build())
+                .getBody().get(0);
 
-            distributionSetTagResource.assignDistributionSets(tag.getTagId(),
-                    sets.stream().map(
-                            set -> new MgmtAssignedDistributionSetRequestBody().setDistributionSetId(set.getDsId()))
-                            .collect(Collectors.toList()));
-        }
+        distributionSetTagResource.assignDistributionSets(tag.getTagId(),
+                sets.stream()
+                        .map(set -> new MgmtAssignedDistributionSetRequestBody().setDistributionSetId(set.getDsId()))
+                        .collect(Collectors.toList()));
     }
 
     private void assignSoftwareModulesTo(final Scenario scenario, final List<MgmtDistributionSet> sets) {
@@ -379,8 +387,10 @@ private List<MgmtSoftwareModule> addModules(final Scenario scenario, final MgmtD
 
     private void createTargets(final Scenario scenario, final List<Long> deviceGroupTags) {
         LOGGER.info("Creating {} targets", scenario.getTargets());
-        IntStream.range(0, scenario.getTargets() / PAGE_SIZE).parallel()
-                .forEach(i -> createTargetPage(scenario, i, deviceGroupTags));
+        final BigDecimal pages = new BigDecimal(scenario.getTargets())
+                .divide(new BigDecimal(PAGE_SIZE), BigDecimal.ROUND_UP).max(new BigDecimal(1));
+
+        IntStream.range(0, pages.intValue()).parallel().forEach(i -> createTargetPage(scenario, i, deviceGroupTags));
         LOGGER.info("Creating {} targets -> Done", scenario.getTargets());
     }
 

diff --git a/hawkbit-core/src/main/java/org/eclipse/hawkbit/repository/TagFields.java b/hawkbit-core/src/main/java/org/eclipse/hawkbit/repository/TagFields.java
@@ -14,6 +14,11 @@
  *
  */
 public enum TagFields implements FieldNameProvider {
+    /**
+     * The id field.
+     */
+    ID("id"),
+
     /**
      * The name field.
      */

diff --git a/hawkbit-core/src/main/java/org/eclipse/hawkbit/repository/TargetFilterQueryFields.java b/hawkbit-core/src/main/java/org/eclipse/hawkbit/repository/TargetFilterQueryFields.java
@@ -34,7 +34,6 @@ public enum TargetFilterQueryFields implements FieldNameProvider {
      */
     AUTOASSIGNDISTRIBUTIONSET("autoAssignDistributionSet", "name", "version");
 
-
     private final String fieldName;
     private List<String> subEntityAttributes;
     private boolean mapField;

diff --git a/...resource/src/test/java/org/eclipse/hawkbit/ddi/rest/resource/DdiArtifactDownloadTest.java b/...resource/src/test/java/org/eclipse/hawkbit/ddi/rest/resource/DdiArtifactDownloadTest.java
@@ -161,7 +161,7 @@ public void invalidRequestsOnArtifactResource() throws Exception {
     public void downloadArtifactThroughFileName() throws Exception {
         downLoadProgress = 1;
         shippedBytes = 0;
-        assertThat(softwareManagement.findSoftwareModulesAll(pageReq)).hasSize(0);
+        assertThat(softwareManagement.findSoftwareModulesAll(PAGE)).hasSize(0);
 
         // create target
         final Target target = testdataFactory.createTarget();

diff --git a/...ddi-resource/src/test/java/org/eclipse/hawkbit/ddi/rest/resource/DdiCancelActionTest.java b/...ddi-resource/src/test/java/org/eclipse/hawkbit/ddi/rest/resource/DdiCancelActionTest.java
@@ -126,13 +126,14 @@ public void rootRsCancelAction() throws Exception {
         // Retrieved is reported
 
         List<Action> activeActionsByTarget = deploymentManagement
-                .findActiveActionsByTarget(savedTarget.getControllerId());
+                .findActiveActionsByTarget(PAGE, savedTarget.getControllerId()).getContent();
 
         assertThat(activeActionsByTarget).hasSize(1);
         assertThat(activeActionsByTarget.get(0).getStatus()).isEqualTo(Status.RUNNING);
         final Action cancelAction = deploymentManagement.cancelAction(updateAction.getId());
 
-        activeActionsByTarget = deploymentManagement.findActiveActionsByTarget(savedTarget.getControllerId());
+        activeActionsByTarget = deploymentManagement.findActiveActionsByTarget(PAGE, savedTarget.getControllerId())
+                .getContent();
 
         // the canceled action should still be active!
         assertThat(cancelAction.isActive()).isTrue();
@@ -174,7 +175,8 @@ public void rootRsCancelAction() throws Exception {
                         .contentType(MediaType.APPLICATION_JSON).accept(MediaType.APPLICATION_JSON))
                 .andDo(MockMvcResultPrinter.print()).andExpect(status().isOk());
 
-        activeActionsByTarget = deploymentManagement.findActiveActionsByTarget(savedTarget.getControllerId());
+        activeActionsByTarget = deploymentManagement.findActiveActionsByTarget(PAGE, savedTarget.getControllerId())
+                .getContent();
         assertThat(activeActionsByTarget).hasSize(0);
         final Action canceledAction = deploymentManagement.findAction(cancelAction.getId()).get();
         assertThat(canceledAction.isActive()).isFalse();
@@ -241,22 +243,22 @@ public void rootRsCancelActionFeedback() throws Exception {
         final Action cancelAction = deploymentManagement.cancelAction(updateAction.getId());
         assertThat(deploymentManagement.countActionStatusAll()).isEqualTo(2);
 
-        assertThat(deploymentManagement.findActiveActionsByTarget(savedTarget.getControllerId())).hasSize(1);
+        assertThat(deploymentManagement.findActiveActionsByTarget(PAGE, savedTarget.getControllerId())).hasSize(1);
         mvc.perform(post("/{tenant}/controller/v1/" + TestdataFactory.DEFAULT_CONTROLLER_ID + "/cancelAction/"
                 + cancelAction.getId() + "/feedback", tenantAware.getCurrentTenant())
                         .content(JsonBuilder.cancelActionFeedback(cancelAction.getId().toString(), "proceeding"))
                         .contentType(MediaType.APPLICATION_JSON).accept(MediaType.APPLICATION_JSON))
                 .andDo(MockMvcResultPrinter.print()).andExpect(status().isOk());
 
-        assertThat(deploymentManagement.findActiveActionsByTarget(savedTarget.getControllerId())).hasSize(1);
+        assertThat(deploymentManagement.findActiveActionsByTarget(PAGE, savedTarget.getControllerId())).hasSize(1);
         assertThat(deploymentManagement.countActionStatusAll()).isEqualTo(3);
 
         mvc.perform(post("/{tenant}/controller/v1/" + TestdataFactory.DEFAULT_CONTROLLER_ID + "/cancelAction/"
                 + cancelAction.getId() + "/feedback", tenantAware.getCurrentTenant())
                         .content(JsonBuilder.cancelActionFeedback(cancelAction.getId().toString(), "resumed"))
                         .contentType(MediaType.APPLICATION_JSON).accept(MediaType.APPLICATION_JSON))
                 .andDo(MockMvcResultPrinter.print()).andExpect(status().isOk());
-        assertThat(deploymentManagement.findActiveActionsByTarget(savedTarget.getControllerId())).hasSize(1);
+        assertThat(deploymentManagement.findActiveActionsByTarget(PAGE, savedTarget.getControllerId())).hasSize(1);
         assertThat(deploymentManagement.countActionStatusAll()).isEqualTo(4);
 
         mvc.perform(post("/{tenant}/controller/v1/" + TestdataFactory.DEFAULT_CONTROLLER_ID + "/cancelAction/"
@@ -265,30 +267,30 @@ public void rootRsCancelActionFeedback() throws Exception {
                         .contentType(MediaType.APPLICATION_JSON).accept(MediaType.APPLICATION_JSON))
                 .andDo(MockMvcResultPrinter.print()).andExpect(status().isOk());
         assertThat(deploymentManagement.countActionStatusAll()).isEqualTo(5);
-        assertThat(deploymentManagement.findActiveActionsByTarget(savedTarget.getControllerId())).hasSize(1);
+        assertThat(deploymentManagement.findActiveActionsByTarget(PAGE, savedTarget.getControllerId())).hasSize(1);
 
         // cancellation canceled -> should remove the action from active
-        assertThat(deploymentManagement.findActiveActionsByTarget(savedTarget.getControllerId())).hasSize(1);
+        assertThat(deploymentManagement.findActiveActionsByTarget(PAGE, savedTarget.getControllerId())).hasSize(1);
         mvc.perform(post("/{tenant}/controller/v1/" + TestdataFactory.DEFAULT_CONTROLLER_ID + "/cancelAction/"
                 + cancelAction.getId() + "/feedback", tenantAware.getCurrentTenant())
                         .content(JsonBuilder.cancelActionFeedback(cancelAction.getId().toString(), "canceled"))
                         .contentType(MediaType.APPLICATION_JSON).accept(MediaType.APPLICATION_JSON))
                 .andDo(MockMvcResultPrinter.print()).andExpect(status().isOk());
         assertThat(deploymentManagement.countActionStatusAll()).isEqualTo(6);
-        assertThat(deploymentManagement.findActiveActionsByTarget(savedTarget.getControllerId())).hasSize(1);
+        assertThat(deploymentManagement.findActiveActionsByTarget(PAGE, savedTarget.getControllerId())).hasSize(1);
 
         // cancellation rejected -> action still active until controller close
         // it
         // with finished or
         // error
-        assertThat(deploymentManagement.findActiveActionsByTarget(savedTarget.getControllerId())).hasSize(1);
+        assertThat(deploymentManagement.findActiveActionsByTarget(PAGE, savedTarget.getControllerId())).hasSize(1);
         mvc.perform(post("/{tenant}/controller/v1/" + TestdataFactory.DEFAULT_CONTROLLER_ID + "/cancelAction/"
                 + cancelAction.getId() + "/feedback", tenantAware.getCurrentTenant())
                         .content(JsonBuilder.cancelActionFeedback(cancelAction.getId().toString(), "rejected"))
                         .contentType(MediaType.APPLICATION_JSON).accept(MediaType.APPLICATION_JSON))
                 .andDo(MockMvcResultPrinter.print()).andExpect(status().isOk());
         assertThat(deploymentManagement.countActionStatusAll()).isEqualTo(7);
-        assertThat(deploymentManagement.findActiveActionsByTarget(savedTarget.getControllerId())).hasSize(1);
+        assertThat(deploymentManagement.findActiveActionsByTarget(PAGE, savedTarget.getControllerId())).hasSize(1);
 
         // update closed -> should remove the action from active
         mvc.perform(post("/{tenant}/controller/v1/" + TestdataFactory.DEFAULT_CONTROLLER_ID + "/deploymentBase/"
@@ -297,7 +299,7 @@ public void rootRsCancelActionFeedback() throws Exception {
                         .contentType(MediaType.APPLICATION_JSON).accept(MediaType.APPLICATION_JSON))
                 .andDo(MockMvcResultPrinter.print()).andExpect(status().isOk());
         assertThat(deploymentManagement.countActionStatusAll()).isEqualTo(8);
-        assertThat(deploymentManagement.findActiveActionsByTarget(savedTarget.getControllerId())).hasSize(0);
+        assertThat(deploymentManagement.findActiveActionsByTarget(PAGE, savedTarget.getControllerId())).hasSize(0);
     }
 
     @Test
@@ -325,12 +327,12 @@ public void multipleCancelActionFeedback() throws Exception {
         assertThat(deploymentManagement.countActionStatusAll()).isEqualTo(3);
 
         // 3 update actions, 0 cancel actions
-        assertThat(deploymentManagement.findActiveActionsByTarget(savedTarget.getControllerId())).hasSize(3);
-        assertThat(deploymentManagement.findActiveActionsByTarget(savedTarget.getControllerId())).hasSize(3);
+        assertThat(deploymentManagement.findActiveActionsByTarget(PAGE, savedTarget.getControllerId())).hasSize(3);
+        assertThat(deploymentManagement.findActiveActionsByTarget(PAGE, savedTarget.getControllerId())).hasSize(3);
         final Action cancelAction = deploymentManagement.cancelAction(updateAction.getId());
         final Action cancelAction2 = deploymentManagement.cancelAction(updateAction2.getId());
 
-        assertThat(deploymentManagement.findActiveActionsByTarget(savedTarget.getControllerId())).hasSize(3);
+        assertThat(deploymentManagement.findActiveActionsByTarget(PAGE, savedTarget.getControllerId())).hasSize(3);
         assertThat(deploymentManagement.countActionsByTarget(savedTarget.getControllerId())).isEqualTo(3);
         mvc.perform(get("/{tenant}/controller/v1/" + TestdataFactory.DEFAULT_CONTROLLER_ID + "/cancelAction/"
                 + cancelAction.getId(), tenantAware.getCurrentTenant()).accept(MediaType.APPLICATION_JSON))
@@ -357,7 +359,7 @@ public void multipleCancelActionFeedback() throws Exception {
         assertThat(deploymentManagement.countActionStatusAll()).isEqualTo(7);
 
         // 1 update actions, 1 cancel actions
-        assertThat(deploymentManagement.findActiveActionsByTarget(savedTarget.getControllerId())).hasSize(2);
+        assertThat(deploymentManagement.findActiveActionsByTarget(PAGE, savedTarget.getControllerId())).hasSize(2);
         assertThat(deploymentManagement.countActionsByTarget(savedTarget.getControllerId())).isEqualTo(3);
         mvc.perform(get("/{tenant}/controller/v1/" + TestdataFactory.DEFAULT_CONTROLLER_ID + "/cancelAction/"
                 + cancelAction2.getId(), tenantAware.getCurrentTenant()).accept(MediaType.APPLICATION_JSON))
@@ -391,12 +393,12 @@ public void multipleCancelActionFeedback() throws Exception {
         assertThat(deploymentManagement.countActionStatusAll()).isEqualTo(10);
 
         // 1 update actions, 0 cancel actions
-        assertThat(deploymentManagement.findActiveActionsByTarget(savedTarget.getControllerId())).hasSize(1);
+        assertThat(deploymentManagement.findActiveActionsByTarget(PAGE, savedTarget.getControllerId())).hasSize(1);
 
         final Action cancelAction3 = deploymentManagement.cancelAction(updateAction3.getId());
 
         // action is in cancelling state
-        assertThat(deploymentManagement.findActiveActionsByTarget(savedTarget.getControllerId())).hasSize(1);
+        assertThat(deploymentManagement.findActiveActionsByTarget(PAGE, savedTarget.getControllerId())).hasSize(1);
         assertThat(deploymentManagement.countActionsByTarget(savedTarget.getControllerId())).isEqualTo(3);
         assertThat(deploymentManagement.getAssignedDistributionSet(TestdataFactory.DEFAULT_CONTROLLER_ID).get())
                 .isEqualTo(ds3);
@@ -418,7 +420,7 @@ public void multipleCancelActionFeedback() throws Exception {
         assertThat(deploymentManagement.countActionStatusAll()).isEqualTo(13);
 
         // final status
-        assertThat(deploymentManagement.findActiveActionsByTarget(savedTarget.getControllerId())).hasSize(0);
+        assertThat(deploymentManagement.findActiveActionsByTarget(PAGE, savedTarget.getControllerId())).hasSize(0);
         assertThat(deploymentManagement.countActionsByTarget(savedTarget.getControllerId())).isEqualTo(3);
     }