feat(ingress): automatic host and path discovery #49

akosveres · 2024-11-04T09:01:20Z

Add ability to create API Checks based on ingress resource host and
path entries
Determine how internal resources are linked together
Add logic to add / update / delete internal ApiCheck resources
Small code improvements
Update docs and samples based on new code changes

internal/controller/networking/ingress_controller.go

akosveres · 2024-11-04T09:10:34Z

Tests are broken, need fixing and update.

internal/controller/networking/ingress_controller.go

alen-z · 2024-11-04T13:35:44Z

config/samples/network_v1_ingress.yaml

    k8s.checklyhq.com/enabled: "true"
-    k8s.checklyhq.com/path: "/baz"
-    # k8s.checklyhq.com/endpoint: "foo.baaz" - Default read from spec.rules[0].host
-    # k8s.checklyhq.com/success: "200" - Default "200"
+    # k8s.checklyhq.com/endpoint: "foo.baaz" - Default read from spec.rules[*].host
    k8s.checklyhq.com/group: "group-sample"
    # k8s.checklyhq.com/muted: "false" # If not set, default "true"
+    # k8s.checklyhq.com/path: "/baz" - Default read from spec.rules[*].http.paths[*].path
+    # k8s.checklyhq.com/success: "200" - Default "200"


With that it seems we can add only once custom endpoint+path+status. I'd suggest to have one annotation that can define multiple targets. E.g.:

annotations: k8s.checklyhq.com/api_monitor: "200|https://foo.baaz/baz,301|http://redirect.demo"

Even better, maybe include JSON in the value that is directly tied to API.

JSON for flexibility, simple syntax for UX. Should be decided, but I'd move from ability to set only one endpoint.

100% agree that we need something, not sure what the best path forward is, I'd hate to manage JSON inside YAML..

Then we need to see if we can find best simple abstraction of what we want to achieve here and not be slapped in the face because it does not support some basic functionality. I'd still have API reference in mind here to make sure we cover basics.

What do you propose?

Also, what would be the minimum (original) API JSON request for hostname+path? Just to see verbosity. Benefit here is the ability to support those kind of annotations even in Service resources and documentation is already available.

danielpaulus · 2024-11-11T07:50:33Z

Seems like the tests are failing still.
@benben can we start dogfooding this operator? I'd like to give some better feedback on these PRs.

akosveres · 2024-11-11T18:43:36Z

Seems like the tests are failing still. @benben can we start dogfooding this operator? I'd like to give some better feedback on these PRs.

Tests are passing now.

* Add ability to create API Checks based on ingress resource host and path entries * Determine how internal resources are linked together * Add logic to add / update / delete internal ApiCheck resources * Small code improvements

* Updated based on the new code updates

* ghcr.io is getting constant rate limits

* if the resource annotation is no longer present * if the enabled annotation is changed to false

internal/controller/networking/ingress_controller.go

sorccu · 2024-11-20T21:07:41Z

internal/controller/networking/ingress_controller.go

+		_, exists := newApiChecksMap[existingApiCheck.Name]
+		if exists {
+			if existingApiCheck.Spec == newApiChecksMap[existingApiCheck.Name].Spec {
+				logger.Info("ApiCheck data is identical, no need for update", "ApiCheck Name", existingApiCheck.Name)
+			} else {
+				logger.Info("ApiCheck data is not identical, update needed", "ApiCheck Name", existingApiCheck.Name, "old spec", existingApiCheck.Spec, "new spec", newApiChecksMap[existingApiCheck.Name].Spec)
+				updateApiChecks = append(updateApiChecks, newApiChecksMap[existingApiCheck.Name])


I think it would be better to just use the map value you've retrieved instead of constantly accessing the value through the map by name, which is prone to mistakes.

Suggested change

_, exists := newApiChecksMap[existingApiCheck.Name]

if exists {

if existingApiCheck.Spec == newApiChecksMap[existingApiCheck.Name].Spec {

logger.Info("ApiCheck data is identical, no need for update", "ApiCheck Name", existingApiCheck.Name)

} else {

logger.Info("ApiCheck data is not identical, update needed", "ApiCheck Name", existingApiCheck.Name, "old spec", existingApiCheck.Spec, "new spec", newApiChecksMap[existingApiCheck.Name].Spec)

updateApiChecks = append(updateApiChecks, newApiChecksMap[existingApiCheck.Name])

newApiCheck, exists := newApiChecksMap[existingApiCheck.Name]

if exists {

if existingApiCheck.Spec == newApiCheck.Spec {

logger.Info("ApiCheck data is identical, no need for update", "ApiCheck Name", existingApiCheck.Name)

} else {

logger.Info("ApiCheck data is not identical, update needed", "ApiCheck Name", existingApiCheck.Name, "old spec", existingApiCheck.Spec, "new spec", newApiCheck.Spec)

updateApiChecks = append(updateApiChecks, newApiCheck)

Fixed in latest commit.

sorccu · 2024-11-20T21:13:50Z

internal/controller/networking/ingress_controller.go

+			continue
+		}
+
+		logger.Info("ApiCheck resource deleted successfully.", apiCheckResource.Name, "Namespace:", apiCheckResource.Namespace)


Super nitpicky but it would be nice to use a consistent format for log messages. Either always end in punctuation or never do. Also I don't think it's necessary to include the : in any log key names.

Fixed in latest commit.

sorccu · 2024-11-20T21:14:19Z

internal/controller/networking/ingress_controller.go

+			continue
+		}
+
+		logger.Info("ApiCheck resource is present, we need to delete it..", "Ingress Name", ingress.Name, "Ingress namespace", ingress.Namespace)


Nitpick but you have double punctuation at the end here.

Fixed in latest commit.

sorccu · 2024-11-20T21:20:26Z

internal/controller/networking/ingress_controller.go

+	labels := make(map[string]*string)
+	labels["ingress-controller"] = &ingress.Name


This labels variable doesn't seem to be used for anything?

It is used when we create the check on checklyhq.com, all labels are added as tags, it helps identifying in the UI where the checks are coming from and in this case, you could see from the tags that it was generated from a specific ingess.

No, I mean that the variable is literally unused. The label map is created, and then a value is set, but then the labels variable is not used for anything, so there is no way for it to end up in the UI. Frankly I'm a little surprised that Go doesn't complain about the variable being unused.

sorccu · 2024-11-20T21:27:51Z

internal/controller/networking/ingress_controller.go

+		var host string
+		if ingress.Annotations[annotationEndpoint] == "" {
+			host = rule.Host
+		} else {
+			host = ingress.Annotations[annotationEndpoint]
+		}


So this pattern seem to be quite prevalent in the codebase, if you prefer it the way it is then alright, but I would probably just do this instead:

Suggested change

var host string

if ingress.Annotations[annotationEndpoint] == "" {

host = rule.Host

} else {

host = ingress.Annotations[annotationEndpoint]

}

host := ingress.Annotations[annotationEndpoint]

if host == "" {

host = rule.Host

}

Always good if you're able to reduce redundant map accesses because it gives you less opportunity to use the wrong key.

Fixed in latest commit.

sorccu · 2024-11-20T21:37:35Z

internal/controller/networking/ingress_controller.go

+	_, checklyAnnotationExists := ingress.Annotations[annotationEnabled]
+	if (!checklyAnnotationExists) || (ingress.Annotations[annotationEnabled] == "false") {


Seems kinda complex. Given how the exists variable is single use here, you might as well just put them in the if's scope. That way you can give them short generic names and it's not going to pollute entire scope. Also, minus one map access (my favorite).

Suggested change

_, checklyAnnotationExists := ingress.Annotations[annotationEnabled]

if (!checklyAnnotationExists) || (ingress.Annotations[annotationEnabled] == "false") {

if value, exists := ingress.Annotations[annotationEnabled]; !exists || value == "false" {

Fixed in latest commit.

internal/controller/networking/ingress_controller.go

sorccu · 2024-11-20T21:51:17Z

internal/controller/networking/ingress_controller.go

-		},
-		Spec: apiCheckSpec,
+	// Update API checks
+	if len(updateApiChecks) > 0 {


You don't really need these len checks, the for ... range will work just fine without it. They're not hurting anything but you'd be able to reduce the nesting level which would make the code a bit easier to read.

Fixed in latest commit.

internal/controller/networking/ingress_controller.go

sorccu · 2024-11-20T22:02:12Z

internal/controller/networking/ingress_controller.go

+			for _, rulePath := range rule.HTTP.Paths {
+				if ingress.Annotations[annotationPath] == "" {
+					if rulePath.Path == "" {
+						paths = append(paths, "/")
+					} else {
+						paths = append(paths, rulePath.Path)
+					}
+				} else {
+					paths = append(paths, ingress.Annotations[annotationPath])
+				}
+			}


I'm not sure if the logic here is correct. It might be, but it seems a little fishy. If there are multiple rule.HTTP.Paths, then you'll iterate over them, but then if the annotationPath exists, you add the same value (annotationPath's value) to paths in every iteration? Would that not basically just create the exact same check multiple times?

That is correct. I guess the right thing to do is to completely remove the path annotation. Do you agree?

Since the path annotation used to be required, removing it would be a breaking change. I would rather not do that unless there is clear, worthy benefit, and I don't think there is (to be clear the feature in the PR is great, just saying I don't think removing the annotation is warranted). Would the following make sense instead:

If ingress.Annotations[annotationPath] exists, then we simply append its value to paths without looping through rule.HTTP.Paths at all. And if it is not set, then we do loop through rule.HTTP.Paths like now, except with all the annotation checking removed.

Thoughts?

sorccu · 2024-12-10T17:44:20Z

internal/controller/networking/ingress_controller.go

+			} else {
+				logger.Info(
+					"ApiCheck data is not identical, update needed", "ApiCheck Name", existingApiCheck.Name, "old spec", existingApiCheck.Spec, "new spec", newApiCheck.Spec)
+				updateApiChecks = append(updateApiChecks, newApiChecksMap[existingApiCheck.Name])


Suggested change

updateApiChecks = append(updateApiChecks, newApiChecksMap[existingApiCheck.Name])

updateApiChecks = append(updateApiChecks, newApiCheck)

akosveres commented Nov 4, 2024

View reviewed changes

internal/controller/networking/ingress_controller.go Outdated Show resolved Hide resolved

alen-z reviewed Nov 4, 2024

View reviewed changes

internal/controller/networking/ingress_controller.go Outdated Show resolved Hide resolved

alen-z reviewed Nov 4, 2024

View reviewed changes

internal/controller/networking/ingress_controller.go Show resolved Hide resolved

alen-z reviewed Nov 4, 2024

View reviewed changes

akosveres mentioned this pull request Nov 6, 2024

Logging verbosity and options #50

Open

akosveres force-pushed the ingress_update branch from 6f1d0cc to 96bb2e4 Compare November 11, 2024 18:33

akosveres force-pushed the ingress_update branch from 3117305 to d48aa93 Compare November 11, 2024 18:54

akosveres added 4 commits November 18, 2024 11:37

feat(ingress): iterate over hosts and paths

cc2fe44

* Add ability to create API Checks based on ingress resource host and path entries * Determine how internal resources are linked together * Add logic to add / update / delete internal ApiCheck resources * Small code improvements

docs(ingress): Update docs and sample

b07f082

* Updated based on the new code updates

build(trivy): use ECR repository

94d2d80

* ghcr.io is getting constant rate limits

build(gha): add smenatic-validate PR step

dfad553

akosveres force-pushed the ingress_update branch from d48aa93 to dfad553 Compare November 18, 2024 11:37

fix(annotations): handle the resource disable state

1fdaac6

* if the resource annotation is no longer present * if the enabled annotation is changed to false

sorccu reviewed Nov 20, 2024

View reviewed changes

internal/controller/networking/ingress_controller.go Outdated Show resolved Hide resolved

sorccu reviewed Nov 20, 2024

View reviewed changes

internal/controller/networking/ingress_controller.go Show resolved Hide resolved

sorccu reviewed Nov 20, 2024

View reviewed changes

internal/controller/networking/ingress_controller.go Show resolved Hide resolved

sorccu reviewed Nov 20, 2024

View reviewed changes

chore(format): quality of code improvements

ccb8805

sorccu reviewed Dec 10, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(ingress): automatic host and path discovery #49

feat(ingress): automatic host and path discovery #49

akosveres commented Nov 4, 2024

akosveres commented Nov 4, 2024 •

edited

Loading

alen-z Nov 4, 2024

akosveres Nov 5, 2024

alen-z Nov 5, 2024

danielpaulus commented Nov 11, 2024

akosveres commented Nov 11, 2024

sorccu Nov 20, 2024

akosveres Nov 25, 2024

sorccu Nov 20, 2024

akosveres Nov 25, 2024

sorccu Nov 20, 2024

akosveres Nov 25, 2024

sorccu Nov 20, 2024

akosveres Nov 25, 2024

sorccu Dec 10, 2024

sorccu Nov 20, 2024

akosveres Nov 25, 2024

sorccu Nov 20, 2024

akosveres Nov 25, 2024

sorccu Nov 20, 2024

akosveres Nov 25, 2024

sorccu Nov 20, 2024

akosveres Nov 25, 2024

sorccu Dec 10, 2024

sorccu Dec 10, 2024

		labels := make(map[string]*string)
		labels["ingress-controller"] = &ingress.Name

		_, checklyAnnotationExists := ingress.Annotations[annotationEnabled]
		if (!checklyAnnotationExists) \|\| (ingress.Annotations[annotationEnabled] == "false") {

	_, checklyAnnotationExists := ingress.Annotations[annotationEnabled]
	if (!checklyAnnotationExists) \|\| (ingress.Annotations[annotationEnabled] == "false") {
	if value, exists := ingress.Annotations[annotationEnabled]; !exists \|\| value == "false" {

	updateApiChecks = append(updateApiChecks, newApiChecksMap[existingApiCheck.Name])
	updateApiChecks = append(updateApiChecks, newApiCheck)

feat(ingress): automatic host and path discovery #49

Are you sure you want to change the base?

feat(ingress): automatic host and path discovery #49

Conversation

akosveres commented Nov 4, 2024

akosveres commented Nov 4, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

danielpaulus commented Nov 11, 2024

akosveres commented Nov 11, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

akosveres commented Nov 4, 2024 •

edited

Loading