Let UNIX agents execute sections/plugins/local checks in parallel

155 votes

As systems are getting more complex and larger, it becomes increasingly difficult to keep the agent's runtime below 60 seconds, as each check is unnecessarily processed sequentially.

In bigger env. it's very likely that you will reach your limits just by having 2-3 plugins running on the same host.

Here are a few runtimes from real customer environments:
35s cmk_agent
25s mk_docker
20s mk_mysql
20s mk_postgres
25s mk_logwatch
90s mk_oracle (even in parallel mode which is supported inside the plugin itself)
60s mk_inventory (big Hardware Unix Systems)
40s mk_filestats

Simply running everything asynchronously is only a makeshift solution. On larger systems, starting multiple checks in parallel is not a problem, as they are designed to handle large loads.

Give the customer the option to choose running the Checks in parallel (wait for Result), async (don't wait) or sequentially (default) and with different Intervals and Timeouts.
Make the agent run only as long as the slowest plugin

Under consideration Checks&Agents Suggested by: Daniel Roettgermann (07 Jun, '22) • Upvoted: yesterday • Comments: 15

Comments: 15

13 Jun, '22
Andrea Kaufmann
Important for a professional environment
1
23 Jun, '22
Lars Sörensen
- add the same basic functions to all unix agents (linux, aix, solaris, hp...), It's really exhausting to ask this for every OS you have again and again.
- add waitmax function to all checks (local, mrpe, plugins, ...) like in the windows agent
2
24 Jun, '22
Mike1098
We fully support this but as mentioned by Lars this should be available for all Agents on all supported OS. Nevertheless the agent must remain smart and efficient. If it consumes to much resources it will not be accepted.

We have the experience that some runtime issues could be fixed on the monitored system.
2
05 Aug, '22
JPH
Oh yes please make all, including Windows, agents consistent in all behaviours an general settings and loggings!
Agents has to be handled all together not per OS.
2
15 Oct, '22
Lars Sörensen
In addition to this feature request:
https://features.checkmk.com/suggestions/336976/add-waitmax-in-all-unix-agents-to-the-local-mrpe-plugin-and-run_cached-checks
14 Aug, '23
Florian
A clear upvote! We have some checks running (especially mk_sap) which we have divided into several plugins, just here we urgently need the parallel execution!
1
12 Oct, '23
Christian
Same at our site! mk_sap, mk_sap_hana, mk_oracle are running very long. Parallel execution would be very, very appreciated.
2
19 Dec, '23
Lars Sörensen
Parallel execution would also have a positive effect on the core in large environments, as the core would have to wait much less time for the agent to finish.

I think this Feature could be accomplished relatively easily by copying the run_cached functionality and adapting it accordingly so that the checks are simply executed in parallel. The new function could be extended with a configurable delay between the start of individual checks so that the server is not stressed too much and a configurable timeout per server so that long runners do not block the agent as is the case today.
1
20 Dec, '23
Niklas Pulina Admin
Hi Lars, thanks a lot for your suggestion. We will take it into consideration when we start working on this functionality.
1
18 Sep, '24
Florian
any news about this topic?
19 Sep, '24
Mohamed Saleh Admin
Hi Florian,

Thank you for following up.
This idea is marked Under consideration, which means we have no plans to implement it as of now.
Will keep you posted with any news.
1
16 Mar
Lars Sörensen
@Mohamed Saleh/Niklas Pulina
It is certainly frustrating when strongly supported ideas remain ignored for years. While we understand the challenges posed by limited resources and competing priorities, it is hard to comprehend why many other ideas were implemented during this period, which required significantly more time and offered less benefit to the majority of customers.

When such ideas are not addressed in a reasonable timeframe despite clear demand, it raises the question of why anyone should continue voting for suggestions. The lack of transparency and communication about why certain ideas are not being considered only serves to undermine trust in the feedback system.
25 Mar
Mohamed Saleh Admin
Hi Lars,

We understand your frustration when an idea that many people support has not yet been implemented. The Ideas Portal is a valuable resource for us. We use it to discover new ideas, understand user needs, and plan future features. However, just because a lot of users vote for an idea doesn't mean it will definitely be implemented. While we do take user votes into account, we must also consider many factors, including strategic direction, technical feasibility, and overall impact on the product. Some ideas may align with future plans, while others help us better understand related challenges that we aim to address in a broader context. Our roadmap is influenced by a combination of market research, strategic priorities, and user feedback. The Ideas Portal is an important part of this process, but it is not the only source of input --> read on the next comment
25 Mar
Mohamed Saleh Admin
continued: Even if we don't implement an idea right away, it still helps shape how our product evolves. It’s worth noting that we’ve released 78 ideas from the portal so far, and more will follow. We appreciate the time and effort our community puts into sharing ideas, voting, and discussing. Your input helps shape Checkmk, and we can’t do it without you!
01 Apr
Lars Sörensen
Hi Mohamed,

Thank you for your response. I understand that the decision-making process involves numerous factors, but increased transparency regarding why certain ideas are not included in the roadmap would be greatly appreciated — especially when these ideas are widely regarded as beneficial.

Agent timeouts and long-running agents have been a recurring issue for us since we began using Checkmk 1.5. This matter has been addressed on numerous occasions over the years and continues to be a concern for us.

I truly value the work you and your team are doing and hope that ideas like mine will be considered in the future. I’d be happy to discuss this further if needed.

Best regards,
Lars
1