Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-arch/arm64 support #672

Merged
merged 4 commits into from
Nov 15, 2024
Merged

Conversation

Blefish
Copy link
Contributor

@Blefish Blefish commented Aug 1, 2024

This PR consists of several small changes:

  • Replaced com.spotify:docker-maven-plugin with io.fabric8:docker-maven-plugin
    Old library is deprecated and does not have buildx support
  • Modified GHA to build and push in the same job stage
    Needed for buildx built images
  • Added buildx and QEMU actions to GHA
    Needed for multi-arch Docker images

I did a brief test on my personal repo by changing GHA code a bit to publish to personal registry and to see if :latest tagging still works as expected. Also ran this on my x86_64 laptop and additionally a t4g.small (aarch64) VM.
Code for the test: https://github.com/Blefish/kafdrop/tree/multi-arch-test
And the test images: https://github.com/Blefish/kafdrop/pkgs/container/kafdrop

Closes #627

@Blefish Blefish mentioned this pull request Aug 1, 2024
@Bert-R
Copy link
Collaborator

Bert-R commented Aug 3, 2024

@Blefish Thanks for this contribution! I need to take a close look at this, but I do not have the time now due to private circumstances. It might take a month or so before I get back on this.

@Bert-R Bert-R self-assigned this Aug 3, 2024
@maipal-c
Copy link

maipal-c commented Sep 7, 2024

@Bert-R any update on this!

@maipal-c
Copy link

maipal-c commented Sep 7, 2024

@Blefish @Bert-R By the way when i teried kafdrop with image - ghcr.io/blefish/kafdrop:latest

ERROR LOGS -


2024-09-07 08:06:21.299 ERROR 1 [  XNIO-1 task-2] i.u.s.a.LoggingExceptionHandler          : UT005023: Exception handling request to /error

java.lang.RuntimeException: jakarta.servlet.ServletException: Request processing failed: freemarker.core.InvalidReferenceException: The following has evaluated to null or missing:
==> error.error  [in template "error.ftlh" at line 3, column 22]

----
Tip: It's the step after the last dot that caused this error, not those before it.
----
Tip: If the failing expression is known to legally refer to something that's sometimes null or missing, either specify a default value like myOptionalVar!myDefault, or use <#if myOptionalVar??>when-present<#else>when-missing</#if>. (These only cover the last step of the expression; to cover the whole expression, use parenthesis: (myOptionalVar.foo)!myDefault, (myOptionalVar.foo)??
----

----
FTL stack trace ("~" means nesting-related):
	- Failed at: #macro header title  [in template "lib/template.ftlh" in macro "header" at line 17, column 1]
	- Reached through: @template.header "${error.error}"  [in template "error.ftlh" at line 3, column 1]
----
	at io.undertow.servlet.spec.HttpServletResponseImpl.doErrorDispatch(HttpServletResponseImpl.java:170)
	at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:283)
	at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135)
	at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:132)
	at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
	at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
	at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:256)
	at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:101)
	at io.undertow.server.Connectors.executeRootHandler(Connectors.java:393)
	at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:859)
	at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538)
	at org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1282)
	at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: jakarta.servlet.ServletException: Request processing failed: freemarker.core.InvalidReferenceException: The following has evaluated to null or missing:
==> error.error  [in template "error.ftlh" at line 3, column 22]

PLATEFORM INFO -

OS- linux (arm64)
OS Image - Amazon Linux 2023.5.20240819
Kernel version- 6.1.102-111.182.amzn2023.aarch64
Container runtime- containerd://1.7.17-k3s1
Kubelet version- v1.29.7+k3s1
aws instance type - r8g.medium

DEPLOYMENT CONFIG -

apiVersion: apps/v1
kind: Deployment
metadata:
   name: kafdrop
   labels:
      app.kubernetes.io/name: kafdrop
spec:
   replicas: 1
   selector:
      matchLabels:
         app.kubernetes.io/name: kafdrop
   template:
      metadata:
         labels:
            app.kubernetes.io/name: kafdrop
      spec:
         serviceAccountName: kafdrop
         automountServiceAccountToken: true
         initContainers:
            - name: attach-init-secrets
              image: redhat/ubi8-minimal
              imagePullPolicy: IfNotPresent
              volumeMounts:
                 - name: init-secrets
                   mountPath: /init-secrets
              command:
                 - "/bin/bash"
                 - "-c"
                 - |
                    JWT=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token) ; 
                    vault_token=$(curl -s --request POST --data '{"jwt": "'$JWT'", "role": "core-kafka"}' http://vault-server-headless.security:8200/v1/auth/in-cluster/login | grep -oP '"client_token":"\K[^"]*') ;
                    secrets=$(curl -s --header "X-Vault-Token: $vault_token" http://vault-server-headless.security:8200/v1/in-cluster/data/core/kafka) ;
                    client_password=$(echo $secrets | grep -oP '"KAFKA_CLIENT_PASSWORDS":"\K[^"]*') ;
                    if [ -z "$client_password" ]; then
                       echo "Failed to get client password from vault. Please update core-kafka role in kubernetes auth method" ;
                       exit 1 ;
                    fi ;
                    cat <<EOF > /init-secrets/broker.properties
                    security.protocol=SASL_PLAINTEXT
                    sasl.mechanism=SCRAM-SHA-256
                    sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="user1" password="$client_password";
                    EOF
         containers:
            - name: kafdrop
              image: ghcr.io/blefish/kafdrop:latest
              imagePullPolicy: IfNotPresent
              ports:
                 - name: http
                   containerPort: 80
                   protocol: TCP
              resources:
                 limits:
                    cpu: 500m
                    memory: 512Gi
                 requests:
                    cpu: 10m
                    memory: 128Mi
              env:
                 - name: SERVER_PORT
                   value: "80"
                 - name: KAFKA_BROKERCONNECT
                   value: "kafka-controller-0.kafka-controller-headless.core.svc.cluster.local:9092"
                 - name: KAFKA_PROPERTIES_FILE
                   value: "/tmp/init-secrets/broker.properties"
                 - name: SERVER_SERVLET_CONTEXTPATH
                   value: "/"
                 - name: JVM_OPTS
                   value: "-Xms32M -Xmx64M"
                 - name: JMX_PORT
                   value: "8686"
                 - name: KAFKA_TRUSTSTORE
                   value: ""
                 - name: KAFKA_KEYSTORE
                   value: ""
                 - name: KAFKA_TRUSTSTORE_FILE
                   value: "kafka.truststore.jks"
                 - name: KAFKA_KEYSTORE_FILE
                   value: "kafka.keystore.jks"
              volumeMounts:
                 - name: init-secrets
                   mountPath: /tmp/init-secrets
                   readOnly: true
              livenessProbe:
                 httpGet:
                    path: /actuator/health
                    port: http
                    scheme: HTTP
                 timeoutSeconds: 5
                 periodSeconds: 10
                 failureThreshold: 5
              readinessProbe:
                 httpGet:
                    path: /actuator/health
                    port: http
                    scheme: HTTP
                 timeoutSeconds: 5
                 periodSeconds: 5
                 failureThreshold: 5
              startupProbe:
                 httpGet:
                    path: /actuator/health
                    port: http
                    scheme: HTTP
                 periodSeconds: 3
                 failureThreshold: 100
         volumes:
            - name: init-secrets
              emptyDir: {}

i am not sure if its code error or image error

@Blefish
Copy link
Contributor Author

Blefish commented Sep 9, 2024

Yup I noticed that too, at random times.

Seemed to happen more often when using SSL security protocol with kafka. It looked to me like some sort of kafka communications error at first glance.

I am not sure, but I think it happened to me as well on the official release, but I need to confirm this.

@maipal-c
Copy link

maipal-c commented Sep 9, 2024

@Blefish when i remove the env variable SERVER_PORT it works fine on default port. your arm image too

Copy link
Collaborator

@Bert-R Bert-R left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again for this contribution and sorry for the delay.
I added one question. Furthermore, I noticed a merge conflict on pom.xml

-
name: Build with Maven
run: mvn -B clean integration-test package assembly:single docker:build
run: mvn -B clean integration-test package assembly:single docker:build docker:push
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pushes snapshot builds to Docker Hub. Why is that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi. I think that in current master, it should also push snapshots and release builds are additionally tagged with :latest.

I think the reason why this is included now within the same build command is due to buildx - it builds the container in an embedded build environment (to emulate arm64) and since it's also wrapped inside docker-maven-plugin, there is no good way to get the built image back to running environment. So buildx is asked to directly push the image.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense. It also makes it easier for people that want to test with a snapshot build.
@davideicardi Do you agree?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it looks good to me. Do you think that we need to change something for release build?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think further changes are needed. The lines below take care of the release process and they still do what happened before: apply a tag of the release and add the latest tag.
I'm going to merge this PR.

* push during build stage as a workaround for buildx
* re-tag the built snapshot if it is a release
@Blefish
Copy link
Contributor Author

Blefish commented Nov 11, 2024

Rebased on latest master

@Bert-R Bert-R merged commit 1b9f882 into obsidiandynamics:master Nov 15, 2024
1 check passed
@Bert-R
Copy link
Collaborator

Bert-R commented Nov 15, 2024

@Blefish Thanks again for this contribution!
I have merged it and the build passed successfully. We now have a snapshot image at Docker hub, with amd64 and arm64 images. If someone can test these images and report back here, that would be great.

@maipal-c
Copy link

@Bert-R i just upgraded to obsidiandynamics/kafdrop:4.0.3-SNAPSHOT (arm image). it worked without any issue in first Go.

  • plateform - kubernetes
  • OS - amazon Linux 2023
  • CPU - aws graviton3 (m7g)

However i felt something weird in logging (formatting) maybe i am wrong-

Screenshot 2024-11-17 at 12 59 24 AM

i'll keep it running. will report here if got any issues.

@davideicardi
Copy link
Collaborator

davideicardi commented Nov 17, 2024

@maipal-c If you are referring to the first weird characters, I suspect it is just the Kafdrop logo not rendered correctly, probably due to the font/spacing.
https://patorjk.com/software/taag/#p=display&f=Alligator2&t=KAFDROP%204

@maipal-c
Copy link

@davideicardi yea it is as issue of lens k8s IDE.

Kafdrop is all good. i am attaching the log file
kafdrop-9d756d49d-5cb6f.log

@Bert-R
Copy link
Collaborator

Bert-R commented Nov 18, 2024

@maipal-c Thanks for the confirmation! @Blefish Compliments to you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

arm64 architecture
4 participants