Commit Graph

34 Commits

Author SHA1 Message Date
Orsiris de Jong
bfa200f3f7
Full Python code fusion / refactor and hardening 2nd edition (#13188)
* New service/discovery/poller wrapper

* Convert old wrapper scripts to bootstrap loaders for wrapper.py

* Move wrapper.py to LibreNMS module directory

* Reformat files

* File reformatting

* bootstrap files reformatting

* Fusion service and wrapper database connections and get_config_data functions

* Moved subprocess calls to command_runner

* LibreNMS library and __init__ fusion

* Reformat files

* Normalize logging use

* Reformatting code

* Fix missing argument for error log

* Fix refactor typo in DBConfig class

* Add default timeout for config.php data fetching

* distributed discovery should finish with a timestamp instead of an epoch

* Fix docstring inside dict prevents service key to work

* Fix poller insert statement

* Fix service wrapper typo

* Update docstring since we changed function behavior

* Normalize SQL statements

* Convert optparse to argparse

* Revert discovery thread number

* Handle debug logging

* Fix file option typo

* Reformat code

* Add credits to source package

* Rename logs depending on the wrapper type

* Cap max logfile size to 10MB

* Reformat code

* Add exception for Redis < 5.0

* Make sure we always log something from service

* Fix bogus description

* Add an error message on missing config file

* Improve error message when .env file cannot be loaded

* Improve wrapper logging

* Fix cron run may fail when environment path is not set

* Add missing -wrapper suffix for logs

* Conform to prior naming scheme

* Linter fix

* Add inline copy of command_runner

* Another linter fix

* Raise exception after logging

* Updated inline command_runner

* Add command_runner to requirements

* I guess I love linter fixes ;)

* Don't spawn more threads than devices

* Fix typo in log call

* Add exit codes to log on error, add command line to debug log

* Add thread name to error message

* Log errors in end message for easier debugging

* Typo fix

* In love of linting
2021-09-27 14:24:25 -05:00
Tony Murray
31246c6ba6
Revert "Full Python code fusion / refactor and hardening (#13094)" (#13123)
This reverts commit 9c534a1a90.
2021-08-10 15:13:05 -05:00
Orsiris de Jong
9c534a1a90
Full Python code fusion / refactor and hardening (#13094)
* Add inline command_runner library

* New service/discovery/poller wrapper

* Convert old wrapper scripts to bootstrap loaders for wrapper.py

* Add command_runner to current requirements

* Move wrapper.py to LibreNMS module directory

* Reformat files

* File reformatting

* bootstrap files reformatting

* Fusion service and wrapper database connections and get_config_data functions

* Moved subprocess calls to command_runner

* LibreNMS library and __init__ fusion

* Reformat files

* Normalize logging use

* Reformatting code

* Fix missing argument for error log

* Fix refactor typo in DBConfig class

* Add default timeout for config.php data fetching

* distributed discovery should finish with a timestamp instead of an epoch

* Fix docstring inside dict prevents service key to work

* Fix poller insert statement

* Fix service wrapper typo

* Update docstring since we changed function behavior

* Normalize SQL statements

* Convert optparse to argparse

* Revert discovery thread number

* Handle debug logging

* Fix file option typo

* Reformat code

* Add credits to source package

* Rename logs depending on the wrapper type

* Cap max logfile size to 10MB

* Reformat code

* Add exception for Redis < 5.0

* Make sure we always log something from service

* Fix bogus description
2021-08-09 18:49:29 -05:00
Jellyfrog
9946fe8b15
Format python code with Black (#12663) 2021-03-28 11:02:33 -05:00
Tony Murray
5f0388f0e3 Remove DB credentials from config.php
Better validation when config.php does not exist

Update docs and quote password

only populate legacy vars in config_to_json
drop .travis.yml config copy
remove credentials from config.php.default

Check for existance of .env instead of config.php in python scripts

legacy credential cleanup

tiny cleanups

consistent env for artisan server and artisan dusk
2020-06-27 07:47:08 -05:00
Tony Murray
2a5b1fd071
Clearer python-memcached info (#11772) 2020-06-04 20:36:25 -05:00
Tony Murray
0b68c70a97
Python wrapper sql cleanup (#11628)
* Clean up wrapp mysql connect a bit

* fix typo
2020-05-19 10:14:09 -05:00
Tony Murray
ec3dc4f89b
Don't use command_runner (#11618) 2020-05-15 08:29:30 -05:00
Orsiris de Jong
1dd0d46edd
Migrate Python scripts to Python 3 (#10759)
* Migrate to python3

* Migrate to python3

* Migrate to python3

* Migrate to python3

* Code refactoring and python 2 compat

* Code refactoring and python 2 compat

* Code refactoring and python 2 compat

* Code refactoring and python 2 compat

* Added shared code for wrappers

* Fix python version check

* Allow pure python MySQL library

* move library.py
remove python2 support bits
remove duplicate code

* fix log location

* whitespace?

* fix pre-existing bug

* fix bug when no devices/services exist

* fix pylint issues

* update imports to match

Co-authored-by: Tony Murray <murraytony@gmail.com>
2020-05-15 00:37:34 -05:00
vsessink
123a669386
fixed race conditions in distributed poller setup (#11307) 2020-04-09 22:38:49 -05:00
Tony Murray
a1aaafa6ff Added ability to enable debug output with wrappers (#8830)
Now with optparse (python 2.3+)

Does not include your doc updates @laf 

If I get some time later, I'll add them, feel free to as well.

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [ ] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
2018-06-20 22:56:51 +01:00
Neil Lathwood
45f98872bb
Revert "Ability to enable debug output with wrappers" (#8829)
Reverts librenms/librenms#8811
2018-06-16 14:32:28 +01:00
Tony Murray
fd77b23624 Added debug options to wrapper scripts to record output to logs/ dir (#8811)
* Ability to enable debug output with wrappers
Output is redirected to a per-device file in the log directory.

* Add warning

* Added some docs to help with debugging using this new option
2018-06-15 14:22:43 +01:00
Tony Murray
a56267e5ba fix: poller-wrapper.py keeps running when a poller takes too log (#7722)
name space the memcache variables for each polling interval based on the rrd.step value.
Make poller-wrapper.py respect the rrd.step setting
2017-11-21 09:22:53 +00:00
Benedict Law
3dab275f14 Fixed poller-wrapper.py so it can connect to database (#7507) 2017-10-18 08:59:36 +01:00
ZedTheYeti
ee5c223669 Supply MySQL database hostname when using Unix sockets to connect (#7471)
* Supply MySQL database hostname when using Unix sockets to connect

* Pass unix_socket parameter when db_socket is set

* Set db_host to localhost when db_socket is set
2017-10-17 21:55:41 -05:00
Tony Murray
b1a414e785 feature: support non-standard unix socket (#5724)
* Add support for custom MySQL unix-socket

* NULL must be lowercase!

* Naive edit of html/install.php

* fixup

* Refactor dbConnect
Use it everywhere

* $config needs to be global
Don't need to set $database_link

* small cleanups
2017-04-06 22:02:37 +01:00
David Bell
5dd030f438 refactor: Added config option for database port (#5517) 2017-01-27 23:16:04 +00:00
Neil Lathwood
7dec670494 refactor: Updated a lot of references to LibreNMS - copyright still left intact (#4339) 2016-09-08 08:12:23 -05:00
Tony Murray
a7408aea88 Spelling correction (#3569) 2016-05-27 10:44:29 -05:00
Karsten Nerdinger
2925d8ce81 Use python2 in Python script shebang lines
This fixes issue #3336.
2016-04-11 15:38:51 +02:00
pblasquez
ab2f1ac394 Update poller-wrapper.py 2016-02-19 18:21:26 -08:00
Paul Gear
b8ad489ff0 Attempted fix for schema update issues
Closes database connection while pollers are running and reopens at the end.
2016-01-21 16:53:57 +10:00
Clint Armstrong
fd0617acca Do not allow the master to join twice
If the poller master doesn't complete in time and ends up with overlapping jobs, neither job will ever complete because both running jobs will think they are the master and wait for the remaining node to exit.
2015-07-02 15:30:57 -04:00
Louis Rossouw
7f3f942e83 Adding the ability to use multiple poller groups for a poller. 2015-06-22 12:37:00 +02:00
Louis Rossouw
611dd07a84 Add logic to poller-wrapper.py to handle non-standard mysql ports in db_host. 2015-06-19 00:22:52 +02:00
Paul Gear
a83cb3d3f7 Fix permissions 2015-05-12 20:13:03 +10:00
Paul Gear
cd74ac8a5a Run flake8 on poller-wrapper; use sensible date format 2015-05-12 19:32:13 +10:00
f0o
cf3eec9bdf Make poller-groups exclusive 2015-03-15 17:19:24 +00:00
f0o
ffd5a3d928 Adds Distributed Polling via Memcached. 2015-03-15 16:29:59 +00:00
laf
3aa4c8b8fc chmod +x on poller-wrapper.py 2014-02-14 07:31:26 +00:00
Paul Gear
1af146e39b Fix branding on poller-wrapper 2013-12-18 03:36:47 +10:00
Paul Gear
58620f5a9d Replace poller-wrapper with exact version from https://github.com/Atrato/observium-poller-wrapper 2013-12-18 03:36:47 +10:00
Tyler Christiansen
b403921398 add poller-wrapper 2013-12-18 03:35:52 +10:00