Commit Graph

353 Commits

Author SHA1 Message Date
Jellyfrog
a70d55bfd8
Add feature to capture a full snmprec (#12706) 2021-04-06 17:26:31 -05:00
Tony Murray
1c379dcd05
Remove legacy function calls (#12651)
* massive inlines

* fix style and wtf

* remove rrdtool.inc.php include

* fix CommonFunctions namespace issues

* looking for missing class space, fix undefined class issues

* style fixes
2021-03-29 00:25:30 +02:00
Jellyfrog
9946fe8b15
Format python code with Black (#12663) 2021-03-28 11:02:33 -05:00
Jellyfrog
f9b25ccdbc
Lint with shellcheck (#12666) 2021-03-28 10:47:21 -05:00
Jellyfrog
b851e140dc
"variant" is now required for test data (#12531)
* Stop snmpsim after we done

* "variant" is now required for test data
2021-02-14 20:39:08 -06:00
Jellyfrog
20b4215204
Swich links to https (#12511)
* Switch librenms links to https

* Convert librenms links in comments

* Switch gnu.org url to https

* switch php urls to https
2021-02-09 00:29:04 +01:00
CrazyMax
4126dddf73
GitHub Actions dev:check ci (#12392)
* Refactor test for GitHub Actions

* Checkout pull request HEAD commit instead of merge commit

* Fetch changed files from API instead of git

* HEAD commit

* Origin

* Fix env

* force full unit checks on gh actions change

* Fix DB conn

* Update test.yml

* Also set port in PDO connection

* Exit with correct exit code

* Update CiHelper.php

* Update test.yml

* Update CiHelper.php

* Update Proc.php

* Cleanups

Remove extra unit test run on php 7.4
Add names that will hopefully show in github.
Remove redundant DB env variables.

* Fix

Shorter names
and DB_TEST env is required

* Change command for information purposes

Co-authored-by: CrazyMax <crazy-max@users.noreply.github.com>
Co-authored-by: Tony Murray <murraytony@gmail.com>
Co-authored-by: Jellyfrog <Jellyfrog@users.noreply.github.com>
2021-01-03 00:54:01 -06:00
RobJE
e1497ecf0b
shellcheck (#12258)
* remove unneeded semi-colons at end of line

* fix SC2206: Quote to prevent word splitting/globbing

* fix SC2164: Use 'cd ... || exit' in case cd fails

* fix SC2004: $/${} is unnecessary on arithmetic variables.

* fix SC2155: Declare and assign separately to avoid masking return values.

* fix SC2124: Assigning an array to a string! Assign as array, or use * instead of @ to concatenate.

* fix SC2059: Don't use variables in the printf format string. Use printf "..%s.." "$foo".

* fix SC2076: Don't quote rhs of =~, it'll match literally rather than as a regex.

* fix SC1090: Can't follow non-constant source. Use a directive to specify location.

* fix SC2166: Prefer [ p ] || [ q ] as [ p -o q ] is not well defined.

* SC2223: This default assignment may cause DoS due to globbing. Quote it.

* fix SC2164: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

* fix SC1117: Backslash is literal in "\n". Prefer explicit escaping: "\\n"

* better handling of branch comapre

* add missing local ver_73 from merge

* remove duplicate definition
2020-11-11 23:21:15 +01:00
Jellyfrog
b7676e4cc8
Update to Composer 2 (#12263)
* Update to Composer 2

No need to roll our own checks anymore since this is now built in.
Also limit the update check to once a week.

* update

* Make Composer wrapper report correct exit code

* Update composer_wrapper.php

* re-enable extension check

Co-authored-by: Tony Murray <murraytony@gmail.com>
2020-11-09 00:28:07 -06:00
Tony Murray
25722bbbd5
Faster initial database creation (#12297)
* Faster initial database creation
Add db dumps to improve initial db migration
On my test system 13s -> 7s, a 46% reduction.
Should help out a lot on systems like RPi

* lnms schema:dump command

--snapshots saves db snapshots, otherwise it saves the yaml

* fix style

* fix timezones...

* not working with in-memory db yet
remove build.sql
2020-11-06 01:19:15 +01:00
Jellyfrog
b866bcf2ab
Revert "Check PHP dependencies in daily.sh (#12260)" (#12271)
This reverts commit d82d2f1f19.
2020-10-29 15:11:32 +01:00
Jellyfrog
d82d2f1f19
Check PHP dependencies in daily.sh (#12260) 2020-10-29 01:34:42 +01:00
Jellyfrog
bd81f76caa
Use variable for PHP min version in docs (#12256)
* Use variable for PHP min version in docs

* Bump mkdocs deps
2020-10-24 11:19:45 +02:00
Jellyfrog
4068b04cc9
Add missing mkdocs extension (#12193)
Regression from 551e95a2e0
2020-10-06 01:52:16 +02:00
Tony Murray
88b40c3d6c
All os detection now uses Yaml (#12186)
* All os detection now uses Yaml
Move OS detection code into the Core module.
Defer all OS with snmp queries.
Add snmpwalk option.
Port ubnt and mobileiron to yaml.

* Silly ide refactor
2020-10-05 14:26:37 +02:00
Birkir Freyr
cb7e191f31
Add panos (paloalto) to rancid_map (#12161)
Add panos (paloalto) to rancid_map
2020-09-28 17:57:54 +02:00
Tony Murray
caf1979529
fix gen_rancid.php permissions (#12153)
broken by #12144
2020-09-24 06:48:14 -05:00
Tony Murray
e944456943
Use lnms dev:check instead of pre-commit.php (#12149)
* Remove pre-commit.php references WIP

* remove pre-commit.php references
2020-09-24 12:32:03 +02:00
Cliff Albert
a85b1e8bbd
scripts/gen_rancid.php: Add support for H3C/Comware (#12144)
* allied, radlan, ciscowlc rancid router.db configuration support
allied and radlan represent allied telesis

* tab/spaces

* allied, radlan, ciscowlc rancid router.db configuration support
allied and radlan represent allied telesis

* * Added rancid support for H3C/Comware

* extraneous newline
2020-09-23 13:35:14 +02:00
Jellyfrog
29f45ca352
Apply fixes from StyleCI (#12124) 2020-09-21 15:59:34 +02:00
Jellyfrog
0d56bbd946
Apply fixes from StyleCI (#12123) 2020-09-21 15:51:53 +02:00
Jellyfrog
77c531527c
Apply fixes from StyleCI (#12117)
* Apply fixes from StyleCI

* Disable style check
2020-09-21 14:54:51 +02:00
Adam Bishop
bbdb1a2a5d
Rewrite smokeping script to be an lnms command (#11585)
* Rewrite smokeping script to be an lnms command

* Add a default number of probes for smokeping

* Formatting fixes

* Refactor to simplify a couple of methods

* Fix a name collision when using more than 60 threads

* Simplify

* First pass at documentation

A few climate fixes too.

I'm a little dubious about the + LibreNMS fix - it could be included
in /etc/smokeping/config if it is really needed, but it sounds like
(looking at git blame) it's needed for if the script generates a broken
configuration file.

I'm thinking we should just not generate broken config.

* Improve testability and add some tests

* Load laravel to make translations available

* Second pass at documentation

* Fix brace

* Extend the device factory to include the type

Also adds a device group fake I created before I realised I didn't need it

* Mimic gen_smokeping.php a little more closely

* Update tests to properly verify old and new behaviour against each other

* Replace gen_smokeping with a wrapper

* Don't double whitespace

Render does this automatically.
Explicitly order by hostname too.

* Make faker less likely to generate duplicates

I tried adding a unique constraint here, but it didn't have the
intended effect.

Extending the hostname like this seems to work fine - I tried
generating 10 million hosts and got no duplicates, compared to
duplicates appearing with as few as 100 hosts without this
 change.

A true fix would be to add an 'fqdn' fake upstream.

* Make the tests more robust

* Assorted bug fixes

* Style corrections

* Handle 'generic' devices

* Fix an issue that came up during rebase

Co-authored-by: Tony Murray <murraytony@gmail.com>
2020-09-18 00:04:54 -05:00
dupondje
ba699be009
Since Rancid 3, the separator is ; (#11688) 2020-09-09 16:10:36 +02:00
Tony Murray
04e7afe9ed
python requirements use sys.exit (#11981)
script was using the wrong exit
2020-07-30 19:05:49 +02:00
Tony Murray
334d50467c
Simpler Python requirements check (#11939)
Don't require the full requirements.txt
If we do that we are likely to send users back to the python2 support branch when new dependencies arise.
2020-07-18 00:12:18 +02:00
Tony Murray
db5524dc09
Merge pull request #11762 from murrant/consolidate-install-doc
Update and Consolidate Install docs Ubuntu 20.04 and CentOS 8
2020-06-29 01:10:28 -05:00
Tony Murray
258d693477 PHP_BINDIR to allow composer.phar to work in more situations 2020-06-27 07:47:08 -05:00
Tony Murray
139317725a one more dep 2020-06-02 22:55:52 -05:00
Tony Murray
a550519c27 full mkdocs deps locked 2020-06-02 22:51:54 -05:00
Tony Murray
ee7fa1e1cb
Change github-remove shebang to python3 (#11753) 2020-06-01 11:24:49 -05:00
Tony Murray
841203c8c5 Consolidated doc (Ubuntu 20.04 Nginx tested) 2020-05-31 21:45:39 -05:00
Tony Murray
2a6342f2ca
Deploy docs direct in Travis CI (#11727)
* deploy_docs direct in travis

* forgot chmod
2020-05-28 21:52:54 -05:00
Tony Murray
46fa4eac03
Add missing bash lint script (#11700) 2020-05-26 08:28:00 -05:00
dupondje
00d00b0fce
Add Edgemax support (#11687) 2020-05-25 17:41:52 +02:00
Tony Murray
ce21011aff
Rewrite development helper to lnms dev:check (#11650)
* Refactor pre-commit to class

* docs build

* dusk check

* ci mode for checks

* full checks

* other mysql

* make other lint checks actually work
fix pylint finding

* ci is a long opt

* fix undefined index

* dusk fully working

* ask for forgiveness, not permission

* fix whitespace

* skip dusk sometimes

* Handle 3com and other os with digits

* flags instead of if else spaghetti

* convert to command

* cleanup

* missed check

* fixes

* case

* self-check :D

* argument now

* fix bugs from refactors

* another fix

* adjust file change parsing

* refactor execut a bit

* fallback to global quiet when unknown type.

* allow quiet override for specific commands

* output cleanup

* check flow

* start of tests

* file categorizer tests and fixes

* fixes and cleanup

* skipable not implemented...

* more tests, fix bugs

* more tests and cleanup

* wrong command

* fix canCheck and set env properly

* full env fix

* don't allow dusk on user's run as it will erase their db.

* fix os option

* fix whitespace

* don't need to start server

* ci doesn't like that
2020-05-22 20:27:48 -05:00
Tony Murray
f1320b6510
Speed up tests by skipping pings (#11642)
* Mock fping for module tests, there is no need.
Removes some wait time.

* fixup some rrd disabling code

* oops
2020-05-19 22:08:41 -05:00
Tony Murray
d5a52ca4eb
Fix various issues with loading os definitions (#11640)
* Ping only device doesn't display
if os was set to something, ping os wasn't loaded and we try to get overview graphs from it.

* Fix snmp_disable device page load error
When other os is set.

* Revamp os setting loading
the only safe way to access is Config::getOsSetting()

* Remove getOsSetting fallback behavior
Most instances don't use it and it can have unexpected results Config::getOsSetting('blah', 'group') == 'librenms'

* refactor and remove unneeded load_os/loadOs calls now since getOsSetting automatically loads it.

* restore unix overview graphs, they are different
small cleanups

* fix
2020-05-19 14:35:32 -05:00
Cliff Albert
576d155613
gen_rancid.php: support for allied telesis and cisco wlc (#11617)
* allied, radlan, ciscowlc rancid router.db configuration support
allied and radlan represent allied telesis

* tab/spaces
2020-05-15 23:47:36 -05:00
TheGreatDoc
1366bdebaf
Added ignored devices to be pinged by smokeping (#11557)
Co-authored-by: Daniel Baeza <d.baeza@tvt-datos.es>
2020-05-14 22:06:18 -05:00
Tony Murray
257d596c3b
Add python validation fix suggestions (#11563)
* add python fixes

* re-order python requirements so most important deps show first.

* Only check dependencies if we are running as the poller user
2020-05-08 09:54:30 -05:00
SourceDoctor
e170529692
Python3 Module Dependency Check (#11544)
* Python3 Modul Dependency Check

* code climate

* binary existance check

* move Python Modul Version Check

* .

* adding verbose argument

* args parser nicement

* enhance Validator for detailed approvements

* Update Python.php

* Update check_requirements.py

* python version to Version class
use Process

Co-authored-by: Tony Murray <murraytony@gmail.com>
2020-05-06 10:58:17 -05:00
Tony Murray
b827250e5d
Install python requirements during daily (#11486)
* install python requirements during daily

* Only check python 3

* ignore failures
2020-05-02 00:30:55 -05:00
Tony Murray
d4225806a8
Run docs tests when only docs are modified. (#11483) 2020-04-25 21:43:03 -05:00
Tony Murray
1c08c11a77
Remove Laravel helpers (#11428)
* Remove Laravel helpers

* Replace qualifier with import
2020-04-18 00:37:56 +02:00
Tony Murray
e4b49f59f6
Delete ports via eloquent event (#11354)
* Delete ports via eloquent event
Chunk delete during purge all operations so we don't use too much memory.

* protect against missing device

* fix whitespace

* fetch less from the database when deleting a device's ports
fix output
2020-04-16 09:19:58 -05:00
Jellyfrog
e111fe690e
Disable xdebug on travis-ci (#11409)
* Disable xdebug on travis-ci

* Update pre-commit.php

Co-authored-by: Tony Murray <murraytony@gmail.com>
2020-04-12 09:11:00 -05:00
Tony Murray
217969e140
Update php-codesniffer (#11368)
* PHPCS 3.5+

* Fix newly detected style issues
2020-04-03 08:41:24 -05:00
Tony Murray
df16de9d2f
Refactor Datastores to allow future improvements. OpenTSDB Tags. (#11283)
* Datastores to object oriented code, using the Laravel IoC container

Change instantiation
better DI
move OpenTSDB

Small re-orgs

remove unused stuff

Fix graphs and other scripts

Use DI for all except rrd

fix up connection error handling

Add tests, fix up a "few" things
Add Config::forget()

Style fixes

Don't reference legacy code

remove accidental code paste

Add datastores phpunit groups

some tests

* rebase fixes

* some test fixes

* shorter tests

* shorter tests

* Don't except when rrdtool can't be started.

* restore tests

* fix rrd tests

* fix iterable change upstream

* fix isValidDataset

* fix invalid data bug

* fix mysql incorrect ds

* fix issue with data that is too long

* use regular data_update()

* Use log facade

* OpenTSDB mis-ordered arguments fix

* Making a singleton with different options makes different singletons.  Just use the global config settings to disable datastores.

* only filter tags for datastores that won't it don't modify the tags permanently

* Update copyrights to include original authors.

* Stats for all datastores

* Fix mysql sends different rrd / other ds names

* fix snmp last stats not initialized
remove unused function

* remove unused function and move single use function closer to its use

* InfluxDB does not need to update null or U values.
Skip write if all fields are empty

* Fix smart value checks

* fix style issues

* Make sure port data is stored the same way as before for Graphite and OpenTSDB
Add ifIndex tag to all to be compatible

* Missed rrdtool_tune() call

* Test update WIP

* OpenTSDB now includes tags

* fix style
2020-03-16 09:17:58 -05:00
Tony Murray
7fe895bd08
Revert "Refactor datastores to classes (#9179)" (#11273)
This reverts commit e5dad7a64e.
2020-03-11 07:52:52 -05:00