starred/netbox
1
0
Fork 0
mirror of https://github.com/netbox-community/netbox.git synced 2026-01-11 21:10:29 +01:00
Code Issues 273 Packages Projects Releases 10 Wiki Activity

Swagger not updating when new API endpoints added in plugin #10477

New Issue
Closed
opened 2025-12-29 21:32:01 +01:00 by adam · 11 comments
adam commented 2025-12-29 21:32:01 +01:00
Owner
Copy Link

Originally created by @tbotnz on GitHub (Nov 15, 2024).

Originally assigned to: @bctiemann on GitHub.

Deployment Type

Self-hosted

Triage priority

N/A

NetBox Version

v4.1.3

Python Version

3.11

Steps to Reproduce

  1. add an endpoint to an API plugin
  2. reload netbox

Expected Behavior

swagger should update showing the new endpoint

Observed Behavior

Problem is caused by the caching of the swagger in redis, this needs to be cleared every time in order for the swagger to update. It does appear to load after the cache expires but this takes 24 hours

Originally created by @tbotnz on GitHub (Nov 15, 2024). Originally assigned to: @bctiemann on GitHub. ### Deployment Type Self-hosted ### Triage priority N/A ### NetBox Version v4.1.3 ### Python Version 3.11 ### Steps to Reproduce 1) add an endpoint to an API plugin 2) reload netbox ### Expected Behavior swagger should update showing the new endpoint ### Observed Behavior Problem is caused by the caching of the swagger in redis, this needs to be cleared every time in order for the swagger to update. It does appear to load after the cache expires but this takes 24 hours
adam added the type: bugstatus: acceptedcomplexity: low labels 2025-12-29 21:32:01 +01:00
adam closed this issue 2025-12-29 21:32:01 +01:00
adam commented 2025-12-29 21:32:02 +01:00
Author
Owner
Copy Link

@Bacloud commented on GitHub (Nov 15, 2024):

Same problem with v4.1.6

@Bacloud commented on GitHub (Nov 15, 2024): Same problem with v4.1.6
adam commented 2025-12-29 21:32:02 +01:00
Author
Owner
Copy Link

@bctiemann commented on GitHub (Nov 15, 2024):

Can this be addressed with documentation, e.g. a note that if you add an API endpoint in your plugin you need to flush your redis cache (with accompanying instructions)?

@bctiemann commented on GitHub (Nov 15, 2024): Can this be addressed with documentation, e.g. a note that if you add an API endpoint in your plugin you need to flush your redis cache (with accompanying instructions)?
adam commented 2025-12-29 21:32:02 +01:00
Author
Owner
Copy Link

@tbotnz commented on GitHub (Nov 15, 2024):

In my mind what should happen is the cached swagger should clear on container/app startup if its deemed out of date

@tbotnz commented on GitHub (Nov 15, 2024): In my mind what should happen is the cached swagger should clear on container/app startup if its deemed out of date
adam commented 2025-12-29 21:32:02 +01:00
Author
Owner
Copy Link

@tbotnz commented on GitHub (Nov 15, 2024):

here a simple management command that fixes it once called python manage.py $command_file_name

$command_file_name.py

from django.core.management.commands.makemigrations import Command as _Command

from django.core.cache import cache


class Command(_Command):

    def handle(self, *args, **kwargs):
        """ Temporary command to clear cache keys that are not cleared by the cache clear command. """
        _KEYS_PREFIX_REQUIRING_CLEARING = [
            "*api_schema_*",
        ]
        for key_prefix in _KEYS_PREFIX_REQUIRING_CLEARING:
            matched_keys_to_delete = cache.keys(key_prefix)
            for matched_key in matched_keys_to_delete:
                cache.delete(matched_key)
@tbotnz commented on GitHub (Nov 15, 2024): here a simple management command that fixes it once called `python manage.py $command_file_name` $command_file_name.py ``` from django.core.management.commands.makemigrations import Command as _Command from django.core.cache import cache class Command(_Command): def handle(self, *args, **kwargs): """ Temporary command to clear cache keys that are not cleared by the cache clear command. """ _KEYS_PREFIX_REQUIRING_CLEARING = [ "*api_schema_*", ] for key_prefix in _KEYS_PREFIX_REQUIRING_CLEARING: matched_keys_to_delete = cache.keys(key_prefix) for matched_key in matched_keys_to_delete: cache.delete(matched_key) ```
adam commented 2025-12-29 21:32:03 +01:00
Author
Owner
Copy Link

@github-actions[bot] commented on GitHub (Nov 23, 2024):

This is a reminder that additional information is needed in order to further triage this issue. If the requested details are not provided, the issue will soon be closed automatically.

@github-actions[bot] commented on GitHub (Nov 23, 2024): This is a reminder that additional information is needed in order to further triage this issue. If the requested details are not provided, the issue will soon be closed automatically.
adam commented 2025-12-29 21:32:03 +01:00
Author
Owner
Copy Link

@prryplatypus commented on GitHub (Nov 26, 2024):

@bctiemann what information is missing here?

@prryplatypus commented on GitHub (Nov 26, 2024): @bctiemann what information is missing here?
adam commented 2025-12-29 21:32:03 +01:00
Author
Owner
Copy Link

@bctiemann commented on GitHub (Nov 27, 2024):

Seems like the ideal solution would be for NetBox to invalidate the cache automatically on startup, but what I'm not clear on is what the mechanism should be for it to determine that the Swagger schema is out of date. How should it "know" that anything has changed?

Alternatively, the manual cache-clearing script above would do the job, but I wonder if it would be more useful for it to just be a general "clear the entire cache" command, with an optional CLI arg to specify only the matching keys to delete (e.g. *api_schema_*), or else CLI args to target specific classes of keys, such as --api_schema. But this seems more like feature territory than a bug fix.

@bctiemann commented on GitHub (Nov 27, 2024): Seems like the ideal solution would be for NetBox to invalidate the cache automatically on startup, but what I'm not clear on is what the mechanism should be for it to determine that the Swagger schema is out of date. How should it "know" that anything has changed? Alternatively, the manual cache-clearing script above would do the job, but I wonder if it would be more useful for it to just be a general "clear the entire cache" command, with an optional CLI arg to specify only the matching keys to delete (e.g. `*api_schema_*`), or else CLI args to target specific classes of keys, such as `--api_schema`. But this seems more like feature territory than a bug fix.
adam commented 2025-12-29 21:32:04 +01:00
Author
Owner
Copy Link

@prryplatypus commented on GitHub (Nov 28, 2024):

Seems like the ideal solution would be for NetBox to invalidate the cache automatically on startup, but what I'm not clear on is what the mechanism should be for it to determine that the Swagger schema is out of date. How should it "know" that anything has changed?

I'm not a Django power user by any means, but what if instead of caching it in Redis across all instances, it was cached in-memory, per instance, during application startup? This would eliminate the need to bust the cache at all, would keep responses to requests quick and would presumably also clear during app restarts (like when using autoreload in development or when deploying new things).

@prryplatypus commented on GitHub (Nov 28, 2024): > Seems like the ideal solution would be for NetBox to invalidate the cache automatically on startup, but what I'm not clear on is what the mechanism should be for it to determine that the Swagger schema is out of date. How should it "know" that anything has changed? I'm not a Django power user by any means, but what if instead of caching it in Redis across all instances, it was cached in-memory, per instance, during application startup? This would eliminate the need to bust the cache at all, would keep responses to requests quick and would presumably also clear during app restarts (like when using autoreload in development or when deploying new things).
adam commented 2025-12-29 21:32:04 +01:00
Author
Owner
Copy Link

@bctiemann commented on GitHub (Dec 6, 2024):

@prryplatypus that actually sounds like how it should work -- but looking at the code now it seems like it's already written so as to cache based on a key that depends on the release version (e.g. 4.1.7):

020386f8a1/netbox/netbox/urls.py (L57-L63)

I think this handles the general use case of a user upgrading from one published version to another, but it doesn't address the case of a developer adding their own API endpoints within a version.

I think developers ought to be able to shoulder clearing the Redis cache manually in this scenario (redis-cli flushall). It seems reasonable to me to just document it accordingly (unless there's confirmation that the caching per-version isn't working?).

I realize (on rereading) that this is already generally known, but it feels like overkill to redesign the whole caching structure just for the sake of developers. (For that same reason I guess the management command approach suggested earlier wouldn't work either, as that would hardly be less work for the developer than just using redis-cli flushall each time they make a code change.)

@bctiemann commented on GitHub (Dec 6, 2024): @prryplatypus that actually sounds like how it should work -- but looking at the code now it seems like it's already written so as to cache based on a key that depends on the release version (e.g. `4.1.7`): https://github.com/netbox-community/netbox/blob/020386f8a143126aee01b5481fbedb9bbc9efc33/netbox/netbox/urls.py#L57-L63 I think this handles the general use case of a user upgrading from one published version to another, but it doesn't address the case of a developer adding their own API endpoints within a version. I think developers ought to be able to shoulder clearing the Redis cache manually in this scenario (`redis-cli flushall`). It seems reasonable to me to just document it accordingly (unless there's confirmation that the caching per-version isn't working?). I realize (on rereading) that this is already generally known, but it feels like overkill to redesign the whole caching structure just for the sake of developers. (For that same reason I guess the management command approach suggested earlier wouldn't work either, as that would hardly be less work for the developer than just using `redis-cli flushall` each time they make a code change.)
adam commented 2025-12-29 21:32:04 +01:00
Author
Owner
Copy Link

@bctiemann commented on GitHub (Dec 10, 2024):

Current proposed solution is to clear the entire Redis cache at startup if DEBUG=True. Please weigh in if it seems that is not a suitable solution.

(We don't want to flush the cache at startup in production situations because many WSGI/gunicorn workers sharing a Redis would be independently recycling on a regular basis and clearing the cache over and over; but for the purposes of development this should save the effort of clearing the cache with every code change.)

@bctiemann commented on GitHub (Dec 10, 2024): Current proposed solution is to clear the entire Redis cache at startup if `DEBUG=True`. Please weigh in if it seems that is not a suitable solution. (We don't want to flush the cache at startup in production situations because many WSGI/gunicorn workers sharing a Redis would be independently recycling on a regular basis and clearing the cache over and over; but for the purposes of development this should save the effort of clearing the cache with every code change.)
adam commented 2025-12-29 21:32:04 +01:00
Author
Owner
Copy Link

@jeremystretch commented on GitHub (Dec 12, 2024):

This seems very reasonable to me, so I'm going to proceed with merging the PR.

@jeremystretch commented on GitHub (Dec 12, 2024): This seems very reasonable to me, so I'm going to proceed with merging the PR.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
beta
breaking change
complexity: high
complexity: low
complexity: medium
needs milestone
netbox
pending closure
plugin candidate
pull-request
Mirrored from GitHub Pull Request
 severity: high
severity: low
severity: medium
status: accepted
status: backlog
status: blocked
status: duplicate
status: needs owner
status: needs triage
status: revisions needed
status: under review
topic: GraphQL
topic: Internationalization
topic: OpenAPI
topic: UI/UX
topic: cabling
topic: event rules
topic: htmx navigation
topic: industrialization
topic: migrations
topic: plugins
topic: scripts
topic: templating
topic: testing
type: bug
type: deprecation
type: documentation
type: feature
type: housekeeping
type: translation
No Label complexity: low status: accepted type: bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
adam
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/netbox#10477
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?