Commit Graph

379 Commits

Author SHA1 Message Date
Tony Murray
e5dad7a64e
Refactor datastores to classes (#9179)
* Datastores to object oriented code, using the Laravel IoC container

Change instantiation
better DI
move OpenTSDB

Small re-orgs

remove unused stuff

Fix graphs and other scripts

Use DI for all except rrd

fix up connection error handling

Add tests, fix up a "few" things
Add Config::forget()

Style fixes

Don't reference legacy code

remove accidental code paste

Add datastores phpunit groups

some tests

* rebase fixes

* some test fixes

* shorter tests

* shorter tests

* Don't except when rrdtool can't be started.

* restore tests

* fix rrd tests

* fix iterable change upstream

* fix isValidDataset

* fix invalid data bug

* fix mysql incorrect ds

* fix issue with data that is too long

* use regular data_update()

* Use log facade

* OpenTSDB mis-ordered arguments fix

* Making a singleton with different options makes different singletons.  Just use the global config settings to disable datastores.

* only filter tags for datastores that won't it don't modify the tags permanently

* Update copyrights to include original authors.

* Stats for all datastores

* Fix mysql sends different rrd / other ds names

* fix snmp last stats not initialized
remove unused function

* remove unused function and move single use function closer to its use

* InfluxDB does not need to update null or U values.
Skip write if all fields are empty

* Fix smart value checks

* fix style issues
2020-03-10 19:08:00 -05:00
SourceDoctor
299da2600e Fix Device Mouseover View (#10962)
* fix device mouseover view
* move function to App/Models/Device.php
* load OS Setting in Device Constructor
* retrigger travis
2019-12-24 17:01:26 +01:00
louis-oui
934260cc75 Feature: Custom OID polling and graphing (#10945)
* merge

* fix db migration

* fix new auth

* fix new auth

* fix new auth

* fix new auth

* fix db schema tests

* fix polling customoid

* fix polling customoid

* fix graph

* fix graph

* fix graph

* fix CI

* fix CI

* always update prev value

* typo
2019-12-18 18:17:21 -06:00
louis-oui
7d9770c9ff Add severity filter to webui alert history (#10918)
* Add severity filter to webui alert history
* Selected options displayed by default
* Fix selected critical
* fix upper
* Fix CI
* fix code climate
2019-12-16 23:58:54 +01:00
Tony Murray
dec9a498ee Updated device cache support (#10795)
* Device Cache
also some additional device related models

* attribs from cache

* replace common attribute functions

* remove legacy cache usage
tidy up some collection manipulation
remove some unused or single use functions

* cleanup some items

* always return a device, to prevent bugs

* clear device cache when testing after each test

* fix double assignment

* Clean up function to take advantage of null object
2019-11-14 21:56:06 +00:00
Tony Murray
e99f421511
Remove legacy auth usage of $_SESSION (#10491)
* Remove auth use of $_SESSION

Will break plugins that depend on $_SESSION, Weathermap was already fixed.
Port them to use Auth::check()/Auth::user()/Auth:id()

* revert accidental replacement
2019-08-05 14:16:05 -05:00
Stuart Henderson
161849219e Avoid unnecessary net-snmp long options for version info (#10405)
Long option support in net-snmp relies on non-portable usage of getopt(3) with "-:" in the argument string, this doesn't work on all OS. Replace with the alternative -V which works everywhere.
2019-07-02 16:52:55 +01:00
Tony Murray
f3ba8947f7
Use Config helper (#10339)
remove usage of global variable
2019-06-23 00:29:12 -05:00
Tony Murray
1a60c44eb0
Device groups rewrite (#10346)
* Device Groups rewrite
Updated web ui
Static or dynamic groups allowed
Alert rule query builder
Translation support
Permissions support

* cleanup, make relationship save, and validate it

* builder WIP

* rules builder and rules saving/loading

* Parse query builder to Laravel Fluent query

* Upgrade existing groups when editing.
Properly update only dynamic groups when polling.

* remove unused old code
Update API and other places to use Eloquent

* debug output in poller restored

* Fix up some things
creating static
improved validation
fix js error on creation
Fix static groups in polling

* hide pattern for static group

* Implement authorization
Use in the menu too

* update schema

* fix rollback

* Don't abort on invalid queries

* fixes to query builder

* add test data, looks like macros aren't handled (omitted them because groups don't use them generally)

* Add macro support for QueryBuilderFluentParser

* add test for macro that accepts value

* More space in forms
Retain rules when converted to static
no duplicate names allowed

* Better error feedback
Update related devices on save

* Add button icon

* format

* update docs

* fix tests

* Fix some QueryBuilderFluentParser issues with OR
updated/more test data

* Show device groups runtime
fix querybuilder.json format

* Store table joins in the rules to minimize polling time
Update group joins in daily.sh (and when they are saved)

* Update daily.php

* Add units to time
2019-06-19 16:01:53 -05:00
Tony Murray
c2b09b8f80
Revert "Rewritten device groups (including static) (#10295)" (#10340)
This reverts commit cfc51d51f5.
2019-06-16 21:52:57 -05:00
Tony Murray
cfc51d51f5
Rewritten device groups (including static) (#10295)
* Device Groups rewrite
Updated web ui
Static or dynamic groups allowed
Alert rule query builder
Translation support
Permissions support

* cleanup, make relationship save, and validate it

* builder WIP

* rules builder and rules saving/loading

* Parse query builder to Laravel Fluent query

* Upgrade existing groups when editing.
Properly update only dynamic groups when polling.

* remove unused old code
Update API and other places to use Eloquent

* debug output in poller restored

* Fix up some things
creating static
improved validation
fix js error on creation
Fix static groups in polling

* hide pattern for static group

* Implement authorization
Use in the menu too

* update schema

* fix rollback

* Don't abort on invalid queries

* fixes to query builder

* add test data, looks like macros aren't handled (omitted them because groups don't use them generally)

* Add macro support for QueryBuilderFluentParser

* add test for macro that accepts value

* More space in forms
Retain rules when converted to static
no duplicate names allowed

* Better error feedback
Update related devices on save

* Add button icon

* format

* update docs

* fix tests
2019-06-16 08:27:22 -05:00
Tony Murray
54e4d0910a
Sanitize report name in pdf.php (#10270)
Authentication users could include arbitrary file.
2019-05-29 09:39:17 -05:00
Tony Murray
36431dd296
Security fix: unauthorized access (#10091)
* Security fix: unauthorized access
Affects nginx users:
Moved php files outside of public html directory (Apache was protected by .htaccess)

Affects all users:
Some files did not check for authentication and could disclose some info.
Better checks before including files from user input

* git mv html/includes/ includes/html
git mv html/pages/ includes/html/
2019-04-11 23:26:42 -05:00
Tony Murray
cb005210d2
Resubmit #9608 (#9941)
* Reorganize trap tests

* Testing db DRIVER to prevent .env from interfering

* New code to detect if Laravel is booted.  Hopefully more reliable.

* WIP external test process

* revert module test helper

* Use .env in Eloquent::boot()

* Fix test database settings loading

* fix undefined classes
(didn't find the one I needed)

* Fix incorrect Config usages
And RrdDefinition return type

* fix .env loading

* use the right DB

* slightly more accurate isConnected

* Move db_name to DBSetupTest specifically

* restore $_SERVER in AuthSSOTest

* missed item

* WIP

* tear down in the correct order.

* some testing cleanups

* remove check for duplicate event listener, it's not working right

* Don't need this change anymore

* Implement Log::event to replace legacy function log_event()

* fix port tests

* fix up tests

* remove pointless TrapTestCase class

* fix style

* Fix db config not being merged...

* skip env check for tests

* defer database operations until after Laravel is booted.

* don't include dbFaciale...

* redundant use
2019-03-12 23:59:03 -05:00
Tony Murray
eb648a1d6f
Revert "Changes to improve testing (#9608)" (#9937)
This reverts commit d1a0ccda46.
2019-03-12 00:49:14 -05:00
Tony Murray
d1a0ccda46
Changes to improve testing (#9608)
* Reorganize trap tests

* Testing db DRIVER to prevent .env from interfering

* New code to detect if Laravel is booted.  Hopefully more reliable.

* WIP external test process

* revert module test helper

* Use .env in Eloquent::boot()

* Fix test database settings loading

* fix undefined classes
(didn't find the one I needed)

* Fix incorrect Config usages
And RrdDefinition return type

* fix .env loading

* use the right DB

* slightly more accurate isConnected

* Move db_name to DBSetupTest specifically

* restore $_SERVER in AuthSSOTest

* missed item

* WIP

* tear down in the correct order.

* some testing cleanups

* remove check for duplicate event listener, it's not working right

* Don't need this change anymore

* Implement Log::event to replace legacy function log_event()

* fix port tests

* fix up tests

* remove pointless TrapTestCase class

* fix style
2019-03-11 22:59:39 -05:00
Tony Murray
8d3a3c9a3e
Handle unexpected os when loading yaml (#9790) 2019-02-06 14:24:39 -06:00
Tony Murray
c4ca7a4f1f
Fix os loading issues (#9763)
* Fix os loading issues

* remove inaccurate debug message
2019-01-30 14:53:43 -06:00
Tony Murray
9b0a427ab0
Update common.php 2019-01-29 07:40:15 -06:00
Tony Murray
7d48c81273 fix snmp debug clipping of quote when obscuring data 2019-01-28 15:41:50 -06:00
Tony Murray
eb59287d34
Fix devices page search & OS loading (#9752)
* Fix devices page search

* add a definition_loaded flag to prevent false positives for already loaded OS

* Fix copy paste error

* and another
2019-01-28 10:37:41 -06:00
Tony Murray
ce6fae8dd1
Rewrite netcmd and ripe whois tools (#9724)
* Port RIPE whois to Laravel

* remove netcmd.php and port to laravel.
Escape CLI and set it to stream output live.

* fix browser buffer bust on a few browsers.
2019-01-25 15:30:58 -06:00
Tony Murray
8ce81f2423
Update database schema display (#9708)
* Update database schema display

* handle missing migrations table
2019-01-22 06:47:07 -06:00
TheGreatDoc
bfcfff6b5e
Merge pull request #9639 from murrant/snmp-exec-timeout
Increase snmp execution time limit to 20 minutes
2019-01-16 13:57:38 +01:00
Tony Murray
d277dee14d
Fix some issues with sensor limits (#9638)
* Fix Zynos temp warn limit
And others, but this seems to be the only one that uses
remove some odd limit restriction in the poller

* More consistent rounding for sensor limits

* round all numeric limits to 10 digits

* update test data

* Remove complex calculation as negative limit values are already swapped as needed.
2019-01-10 19:48:37 -06:00
Tony Murray
561e2fb6e2 Increase snmp execution time limit to 10 minutes
This is needed for discovery in some cases (mostly the arp-table module)
2019-01-10 11:18:34 -06:00
Tony Murray
def8b3e514
Add librenms command (#9619)
* Add librenms command

Hook up to artisan.  Hide dev commands and most other commands if app environment is production.
Register all commands for php artisan or when not in production.

* remove dead end line

* Console application name and version.
Had to shift some stuff from legacy code, but deferred others as it was too extensive of a change.

* switch check order

* always get local version now

* update array format

* whitepace

* fix style
2019-01-08 21:42:56 -06:00
Tony Murray
010ff08e7f
Fix for snmp queries that take longer than 60s (#9574) 2018-12-20 19:51:06 -06:00
PipoCanaja
2d84606abf BUG - correcting size and non significant zeroes (#9575) 2018-12-20 19:50:36 -06:00
Tony Murray
74882e3950 Dashboard widget update (#9515)
Implemented in Laravel
Doesn't use legacy PHP session
Several widgets have new features and settings, for example:

- Multiple ports in one graph
- Maps settings are configurable and override system settings but default to system settings
- Graylog stream and/or device selection
- Much improved graph widget selection
- Many more

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
After you are done testing, you can remove the changes with `./scripts/github-remove`.  If there are schema changes, you can ask on discord how to revert.
2018-12-16 21:18:17 +00:00
Tony Murray
fad0bffc43 Escape SNMP & IPMI shell commands (#9537)
hopefully doesn't break anything
Mostly issues with snmp oids and options containing spaces.  Try to remove all of those.

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
After you are done testing, you can remove the changes with `./scripts/github-remove`.  If there are schema changes, you can ask on discord how to revert.
2018-12-16 13:42:50 +00:00
Tony Murray
8bddfe7225 Fixed plugins using d_echo (#9498)
Move d_echo to helpers.php and include in autoload
Don't remove from common.php yet to be extra safe.
2018-12-08 13:16:49 +00:00
Tony Murray
c7f70b6e47 Added detection if this is a git based install or not. (#9379)
Not sure about the warning or text.

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
After you are done testing, you can remove the changes with `./scripts/github-remove`.  If there are schema changes, you can ask on discord how to revert.
2018-12-07 22:29:37 +00:00
Tony Murray
93f44d67f8 Fixed customers page (#9521)
move customers table backend to Laravel

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
After you are done testing, you can remove the changes with `./scripts/github-remove`.  If there are schema changes, you can ask on discord how to revert.
2018-12-07 22:24:53 +00:00
Tony Murray
3e35ee0e7d Refactored and update Location Geocoding (#9359)
- Fix location so it is a regular database relation (this allows multiple devices to be accurately linked to one location and saves api calls)
- Parse coordinates from the location more consistently
- Add settings to webui
- ~~Used [PHP Geocoder](http://geocoder-php.org/), which has lots of backends and is well tested. (also includes reverse and geoip)~~
- Google Maps, Bing, Mapquest, and OpenStreetMap supported initially.
- Default to OpenStreetMap, which doesn't require a key.  They will liberally hand out bans if you exceed 1 query per second though.
- All other Geocoding APIs require an API key. (Google requires a credit card on file, but seems to be the most accurate)
- Update all (I think) sql queries to handle the new structure
- Remove final vestiges of override_sysLocation as a device attribute
- Update existing device groups and rules in DB
- Tested all APIs with good/bad location, no/bad/good key, and no connection.
- Cannot fix advanced queries that use location

This blocks #8868

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
After you are done testing, you can remove the changes with `./scripts/github-remove`.  If there are schema changes, you can ask on discord how to revert.
2018-11-28 22:49:18 +00:00
PipoCanaja
41cc344106 VRP: Discovery improvement for optical power + Mib upgrade (#9225)
* Huawei Mib file update

* VRP: Improve sensor discovery for optical equipment

* VRP: Improve sensor discovery for optical equipment

* VRP: Adding Test data after sensor dbm update
2018-09-19 10:00:11 -05:00
Tony Murray
32a7c50189
Use Laravel authentication (#8702)
* Use Laravel for authentication
Support legacy auth methods
Always create DB entry for users (segregate by auth method)

Port api auth to Laravel

restrict poller errors to devices the user has access to

Run checks on every page load.  But set a 5 minute (configurable) timer.
Only run some checks if the user is an admin

Move toastr down a few pixels so it isn't as annoying.

Fix menu not loaded on laravel pages when twofactor is enabled for the system, but disabled for the user.
Add two missing menu entries in the laravel menu

Rewrite 2FA code
Simplify some and verify code before applying

Get http-auth working
Handle legacy $_SESSION differently.  Allows Auth::once(), etc to work.

* Fix tests and mysqli extension check

* remove duplicate Toastr messages

* Fix new items

* Rename 266.sql to 267.sql
2018-09-11 07:51:35 -05:00
Tony Murray
a2e33efba8 Use PDO for MySQL queries (#8935)
Gut dbFacile
mres() now does nothing... could have some vulnerable queries, but most are ok

TODO
- [x] IRC client broke
- [x] Install broke

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
2018-08-17 21:29:20 +01:00
Tony Murray
50bb72f530 Remove port association mode map from the database (#8941)
* Remove port association mode map from the database

* Renamed sql file

* Renamed sql file
2018-08-05 09:50:13 +01:00
Tony Murray
25dc44a574 fixed can_ping_device() logic (#8906)
DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
2018-07-17 08:16:13 +01:00
Neil Lathwood
ad6ec7f72a
Refactor alert templates to use Laravel Blade templating engine (#8803)
* Initial start on new templating support

* further updates

* more updates

* More working version

* Last fixes

* Small updates

* modified test to look for dbname.

* Schema update

* Added fix for not being able to disable Philips Hue transport

* Updated sql file

* Updated db_schema

* Set myclabs/deep-copy to be 1.7.x version in composer

* Fixes from murrant

* Forced nikic/php-parser to version 3.1.x in composer

* Updated composer to use custom fork of string-blade-compiler + fixed transport use

* Updated to always use correct template

* Merged legacy and blade templating engines

* Removed template type + fixed test-alert.php

* Added more template placeholders

* Added ability to reference data in templates as $alert->X

* Updated docs for templates

* Updated db_schema.yaml

* Added $alert->builder

* Clarify the use of $alert->builder

* Fixed the use of $alert->transport

* renamed schema file

* Added template validation

* Small update to fix travis issue

* Add Docs text to title bar

* Updated amqp to new of version

* Consistency in alert rule and template title bars
2018-07-14 22:15:43 +01:00
Tony Murray
eeb3d58f5b Improved Logging and Debugging (#8870)
Use Log facility when Laravel is booted.
Update init.php so we can easily boot Laravel for CLI scripts. (and just Eloquent, but that may go away)
Move all debug setup into set_debug() function and use that across all scripts.
Log Laravel database queries.
Send debug output to librenms log file when enabling debug in the webui.
Allow for colorized Log CLI output. (currently will leave % tags in log file output)

** Needs testing and perhaps tweaking still.

DO NOT DELETE THIS TEXT

#### Please note

> Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting.

- [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/)

#### Testers

If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926`
2018-07-13 23:08:00 +01:00
Richard Mayhew
528b91a056 corrected display of minigraph when using sysName as hostname (#8842)
* corrected display of minigraph when using sysName as hostname

* Check to see if its an IP or hostname. Make sure all 3 scenarios work.

* removed test rrd symlink

* removed test rrd symlink

* reverted old change

* Improve and add tests.
2018-07-02 08:23:09 -05:00
Richard Mayhew
ab5d17d715 Allow the hostname to resolve to the sysName, ie Dynamic DNS (#8810) 2018-06-23 21:59:09 +01:00
Tony Murray
9152becec7 refactor: Moved AD user and group filter functions into ActiveDirectoryAuthorizer (#8545) 2018-04-11 10:06:46 +01:00
Tony Murray
7250376104 refactor: Don't access $_SESSION directly for Auth (#8513)
* Don't access $_SESSION directly for Auth

* fix style

* add property annotations
2018-04-07 21:55:28 +01:00
sorribes22
c22b31506e api: Added lattitude and longitude on devices/device fetch (#8515)
* 'lat' and 'lng' columns added on list_devices()

* 'lat' and 'lng' columns added on device fetch

Helped by https://github.com/laf

* Commit user.name changed
2018-04-07 13:29:46 +01:00
Neil Lathwood
03076c4025
feature: Added new alert rule builder UI and rule mapping UI (#8293)
* feature: Added new alert rule builder UI

* Updated to export sql queries

* More updates

* more changes

* removed debug

* fix scrut

* Updated to include import options + various other fixes

* fix rule

* Populate name from collection rules.

* Fix default rule import
Allow new and old style rules in the collection.
Don't add new yet as I'm not sure GenSQL() is working.

* Fix GenSQL call

* Extract filter building to class so it is nicely contained in one place

* moved schema

* some fixes and tweaks

* travis fixes

* Some more features / updates

* Fix up my mistakes when adding default rules

* Use a modal for new alert (Incomplete)
Larger dialog!!
Remove page loading stuff.

Working:
Loading rules, resetting dialog, importing from collection.

Not working yet:
select width
device limited rule access? don't know what this is...

Lots of unused stuff to delete...

* reload "table" after save

* fixed editing rule

* Auto select2 width

* Reload window on save

* Restore per-device alert. Remove debug.

* Small cleanups. Rule Name first.

* Restore button to button type. Rename schema.

* Fixes: wrong command to reload window, remove extra attributes, rule is never passed

* Fixed old rule editing

* some small updates for old imports

* travis update to use trusty

* maybe travis fix

* Ability to set alert rule mappings on the rule edit screen

* pip installs one line, no quiet for deploy

* update schema def

* Fix style and some copyright headers

* fix docs missing file

* Allow new versions of snmpsim and libraries

* Parser WIP

* Fix default rules insert

* reorganize

* Legacy import first draft done

* Implement saving
Skip translation to sql for now

* Working on glues

* small rule collection fix

* Working on glues

* Working on glues

* Docs updates + small UI changes

* Parser WIP

* reorganize

* Legacy import first draft done

* Implement saving
Skip translation to sql for now

* Working on glues

* Working on glues

* Working on glues

* Add table mapping, should move to it's own class

* WIP

* Glue working!!

* Extract Schema class

* Some final touches.
revert alerts_rules.json for now.

* Finish up initial implementation
Needs more tests

* Fix a few places

* small doc updates

* Fix finding tables in grouped rules.

* remove unused code

* code format fixes

* Some quick tests for Schema
Simplified output for findRelationshipPath. Always includes start and target in the result.
This simplifies a lot of code in QueryBuilderParser.php
This also always loads the target table data now (which we want)

* Make bill_id the PRIMARY index for the bills table

* Load macros from a json file in misc instead of the database.

* Fix whitespace and wrong key for collection.

* Handle IN properly when generating SQL

* Fix glue (devices.device_id = ports.port_id) is incorrect :D
Show ALL tables we can resolve relationships for in the query builder filter.

* Remove all macros from the database
Remove insert statements, leave updates to update user's existing rules.
2018-03-14 20:25:19 +00:00
James Andrewartha
70714fe811 security: Hide more confidential data (SNMPv3, IPMI) from the debug output (#8303) 2018-02-27 15:41:44 +00:00
Tony Murray
b2762d9fa7 refactor: Prevent function collisions with Laravel (#8166)
Make function call compatible
drop insensitive starts_with and ends_with for now as they aren't needed.
2018-01-29 21:58:21 +00:00
crcro
7e03905e63 webui: Small graylog cleanup (#8057) 2018-01-09 20:47:33 +00:00
Zmegolaz
2c9468979f Load device attributes before load_os() in device_by_id_cache(). (#8044) 2018-01-08 13:37:34 -06:00
Neil Lathwood
7d0ad5c4ef
tests: Added testing for Arista EOS BGP (#7984)
* tests: Added testing fro Arista EOS BGP

* fixing tests

* small updates

* last one

* more updates

* Account for no space after = in snmp data
always load_os() when a new device is freshly loaded by device_by_id_cache()

* don't be fatal in load_os()

* Update json to correct data
2017-12-30 10:31:06 +00:00
Neil Lathwood
27368a2d98 webui: Stop stripping html tags from snmp creds (#7951) 2017-12-25 09:51:31 -06:00
crcro
520fc4a268 webui: Graylog table refresh (#7832)
* initial release

* style fixes
2017-12-24 19:59:16 +00:00
crcro
71711326fb webui: Syslog table refresh (#7796)
* initial changes

* final changes

* added missing copyright, removed commented old code
2017-12-24 19:57:54 +00:00
Tony Murray
fff66d3c00
Feature: Generic discovery and poller tests (#7873)
* Processor Tests!

* Capture data from live devices easily.

* fix up some stuff, remove powerconnect things as they seem to be just broken.

* formatting, fix missing assignment
add netonix processor data

* fix multi-line, always add sysDescr and sysObjectID
ios cpm test file

* revert composer change and fix whitespace issues

* add help text

* missed help text

* tighter debug output

* handle empty strings properly and mibs with numbers

* use keys for sorting as intended

* fix type with empty data

* oops :)

* whitespace fix

* try installing fping

* Fix TestCase collision + cleanup

* mark TestCase as abstract
don't run two instances of snmpsim

* better database dumps, improved capture

* style fixes

* fix quotes add a few more tables

* add --prefer-new, properly merge data

* Support separate discovery and poller data. But don't waste space if they aren't needed.

* refactor to use class
collects all code in one place for reusability

* reorganize

* Print out when not saving.

* Support for running multiple (or all) modules at once.

* tidy

* Change unit test to a generic name and test all modules we have data for.

* Add documentation and a few more tidies

* whitespace fixes

* Fix tests, add a couple more modules, more docs

* More docs updates

* Fix scrutinizer issues

* add bgp-peers
2017-12-20 08:36:49 -06:00
bergroth
84fdfdf542 Junos dwdm interface sensor support (#7714)
* Junos dwdm interface sensor support  …
  four new sensors
    Chromatic Dispersion ps/nm
    Delay s (seconds)
    Q_factor (dB)
    preFEC_BER (ratio)

  reuse of
    SNR, dbm, temperature, current

* new dwdm sensors more generic, ber and quality_factor

* new sensors updated

* renamed html graphs new sensors

*  fixed ber polling

* Change to base SI Hz and s.

* update sensor divisors to bigint

* rename sql-schema/216.sql to 218.sql

* YAML junos dwmd sensor definitions, except BER

* bigint divisor fixed

* deafult 1 divisior

* small fixes including schema change for another PR

* schema fix
2017-12-12 20:27:10 -06:00
Tony Murray
c22c879983 fix: Remove faulty memcached code (not related to distributed polling) (#7881) 2017-12-10 20:40:45 +00:00
Tony Murray
b620692426 refactor: Update all applications to store metrics (#7853)
* Update all applications to store metrics

* fix db schema

* Add glue test
2017-12-06 22:13:10 +00:00
Tony Murray
5405498ed6 alerts: Adde generic alerts for state sensors (#7812)
* feature: Generic alerts for state sensors
A bit of a hack, but gets the job done. Perhaps someone can refactor the ResolveGlues() function sometime to make it better.

* Fix mistakes

* Add docs and collection rules, critical enabled by default

* moved schema file to 221.sql
2017-12-02 23:03:36 +00:00
Tony Murray
4a03e7838e feature: Save application metrics to db for alerting (#7828)
* feature: save application metrics to db for alerting
However, alerting will not work because ResolveGlues() is broken.
Can add workaround after state_translations alerting is merged

Does not update all applications yet, not sure if that should be done here or in another PR.

Introduces two handy functions dbDeleteOrphans() and array_by_column().  Will replace those in other locations after this is merged or separate them out if this is not merged.

* remove accidental inclusions

* Add db schema
2017-12-01 07:53:26 +00:00
Tony Murray
257a314fca feature: Validate session directory is writable (#7784) 2017-11-23 08:29:10 +00:00
Tony Murray
0dee809607 fix: Discovery os changes weren't properly reflected if os changed (#7779) 2017-11-22 21:48:28 +00:00
Jonathan Lassoff
fe8d011dce refactor: Allow setting the target shorthost length. (#7453)
* Create a target shorthost length. Addresses #7452.

* Update defaults.inc.php
2017-11-09 20:47:52 +00:00
Tony Murray
cbe6cb532d Fix: Discovery validation with ping-only devices (#7665)
* fix: validation check cli php extensions
Fix fatal errors when mysql is not loaded / can't connect

* fix: discovery validation incorrectly detects on ping only devices.
2017-11-07 20:09:49 +00:00
Tony Murray
54b476c769
fix: devices detected as ibmtl or generic (#7618)
* fix: devices detected as ibmtl
Guess at the sysObjectID, please report if you have devices that should be ibmtl and are no longer detected.
Add output of os in discovery.

* Always load all os for discovery.  Should be cached most of the time.
2017-11-03 16:10:24 -05:00
Neil Lathwood
3e94077a71
Revert "Update RFC1123 Hostname Check" (#7577)
* Revert "Revert "feature: Allow snmpget in os discovery yaml (#7558)" (#7576)"

This reverts commit 426916dbe4.

* Revert "fix: lldp find_port_id finds not the most correct port (#7564)"

This reverts commit 7e05fff615.

* Revert "fix: Update RFC1123 Hostname Check (#7572)"

This reverts commit ebcfa45a93.
2017-10-29 20:05:59 +00:00
Parth Laxmikant Kolekar
ebcfa45a93 fix: Update RFC1123 Hostname Check (#7572)
* Update RFC1123 Hostname Check

The regex was wrong. It failed to match devices like `cisco-3750x` even though it is a valid hostname. 
I picked this regex from the RFC1123 documentation, and updated it.

* Update CommonFunctionsTest.php

Update tests

* Updated to allow domains ending with a dot
2017-10-29 11:12:32 +01:00
Zmegolaz
f8d7ccfe0d feature: Support for up/down detection of ping only devices.
* Added support for ping only devices.

* Renamed sql-schema/206 to 207.

* Discovery tried to detect the OS when it shouldn't.

* Updated db_schema.yaml

* Added ping icon.

* Don't show unused graphs and tabs when SNMP is disabled.

* Allow the user to specify OS of ping only devices.

* Removing custom OS now changes it to 'ping'.

* Removed unnecessary use of mres().

* UI select box for SNMP enable/disable.

* Hide device_ping_perf if ping is disable on device.

* Fixed SNMP settings update status messages.

* Added functionality to add ping only devices via the web ui.

* Added ping only option to addhost.php

* Added ping only support to snmp-scan.py

* Moved sql 208 to 211

* Fixed scrutinizer issues.

* Fixed broken ossearch.

* Added ping only support to the API.

* Updated API doc.

* Added (optional) to OS and hardware description. Hid Port Settings, Applications, Modules, Storage, Processors, Memory and Components from the edit menu

* Style fix.

* Updated ping icon.

* clean() instead of mres(). More escaping. Better help in snmp-scan.py and addhost.php

* Fixed scrutinizer issue.

* Always try SNMP in snmp-scan.py, new option for it in addhost.php. Slice instead of chunk in ajax_ossuggest.php. Other minor style changes.

* Updated sql modifications to insert the new column in the same place as in db_schema.yaml.
2017-10-27 22:59:25 -05:00
Tony Murray
8a051aa295 fix: Fix a couple of OS cache invalidation issues (#7567)
* fix: fix a couple of OS cache invalidation issues
Fixes issues where we might be using outdated os definitions.

* use Config and fix style issue

* only pull in settings from config.php (otherwise will possibly contain populated os config)
2017-10-27 21:03:58 +01:00
Tony Murray
51ba934e11 feature: Added web validation (#7474)
* refactor: convert validations to modules to prep for gui integration

* accidentally dropped, schema update

* fix accidental output to webui

* mention discovery-wrapper.py and re-arrange.

* refine some fix statements

* rename the Config validation group to Configuration.

* fix some scrutinizer issues
remove as many local functions from validator.php as possible
move extensions from pre-check
remove duplicate timezone check
looks like there is some db schema differences between mariadb 10.1 and 10.2, investigating

* mariadb version diff take2

* Check schema version first for database.
Remove stop to go back to command line for install docs.
Add helpful link when there is no devices added to /addhost

* fix incorrect validation for empty string defaults

* Fix style

* Add additional file permissions checks

* catch exception and fail for invalid timezone
Change visuals a bit.

* add php version warning

* fix space
2017-10-26 07:56:09 +01:00
Tony Murray
a2b9342f05 fix: improve accuracy of is_valid_hostname() (#7435)
* fix: improve accuracy of is_valid_hostname()
fixes discovery code attempting to add discover invalid dns names
the old code allowed some invalid hostnames, this is more thorough
add tests

* fix merge error
2017-10-05 16:04:17 -05:00
Tony Murray
c0b267b49c refactor: Discovery protocols re-write (#7380)
* refactor: discovery protocols re-write
extract more code to make it easier to read
use snmpwalk_group
walk lldpRemTable instead of lldpRemoteSystemsData
Optimize discover_device to reduce dns lookups of bad dns names
Make discovery_by_ip behavior consistent between cdp and lldp
move can_skip_discovery to discovery functions

Tested lldp, cdp, and ospf.

* always discover links even if the remote device isn't in system
Fix discover_link always updating

* fix style

* fix update, not fetching id

* increase platform length.  Most devices use sysDescr here, which is max length 255

* remove orphaned links

* update db_schema.yaml
2017-10-04 08:16:23 +01:00
Neil Lathwood
f7e14df90e newdevice: Updated Arista entity-physical support to use high/low values from device (#7156)
* fix: Fixed the high/low thresholds for Arista devices

* final updates

* push threshold changes

* more updates

* another update

* updated entity-sensor for current value calc

* hoping final update
2017-09-07 08:49:32 -05:00
Tony Murray
c8436d2a4f
fix: issues with Geist Watchdog
miss-named variable in sensor pre-caching
internal humidity and temperature was discovered twice
humidity was mis-spelled in yaml discovery
temperature and current had incorrect divisor in yaml

remove redundant fahrenheit to celsius function

fixes #7236
2017-08-28 13:00:29 -05:00
Neil Lathwood
25d79ce9d7 newdevice: Added further sensor support for Geist Watchdog (#7143)
* newdevice: Added further sensor support for Geist Watchdog

* relaxed if check
2017-08-19 20:22:23 +01:00
Neil Lathwood
96b010a735 feature: Forced OS Cache rebuild for unit tests (#7163) 2017-08-16 13:58:10 -05:00
Tony Murray
1d777693e1 feature: New IP parsing classes. Removes usage of Pear Net_IPv4 and Net_IPv6. (#7106)
* Refactor remove use of pear Net_IPv4 and Net_IPv6

Caution, I added tests to test the functionality of the new functions, but I'm not 100% certain the replaced usages match 100%.  Please test.

Left usage in generate-iplist.php because I'm not sure the use of that script, so I didn't want to implement additional functionality to support it.

Add support for ipv6 fo ipInNetwork() function

Add phpdocs to functions
Remove accidental inclusion

Use binary operations instead of string for networKFromIp()
Tidy up binary operations in ipInNetwork()

tidy parsing cidr network

Change to a class based implementation.

Update phpdocs.
Fully catch exceptions for hex parsing.

Fix mistake.

support older phpunit versions

Fix php shifting to negative with php 5

* Fix graph authorization. Uncaught exception and checking an ipv4 against an ipv6 network == fail.

* Also remove colons from hex strings.

* Fix typo in bgp polling
Two passes of removing invalid characters from hex strings. (That way we don't mangle valid IPs)
2017-08-08 14:14:58 -05:00
Tony Murray
1e77d4b3ea refactor: validate.php improvements (#6973)
Always output header (may be missing mysql data)
Improve version display, now looks like '1.28-129-g74e6c3edf' <tag>-<commits since tag>-<shortag>
Add mysqli to the list of required extensions, check extensions before init
Add dbIsConnected() function, used to check if we have db in version_info()
Do not die on IPv4 and IPv6 so validate can print errors.
Move git checks so they are output together
fix rrdtool 1.7.0 reports version as 1.7.01.7.0
2017-07-17 19:02:28 +01:00
Aldemir Akpinar
5581c40e19 webui: Order the list of devices in get_all_devices() (#6797) 2017-06-07 14:20:59 +01:00
Tony Murray
b6e18db766 fix: page/graph load speed: part 1 (#6611)
* fix: page/graph load speed: part 1
Speed up load_all_os() by caching the full $config['os'] array
Function time goes from 1.13648s -> 0.00073s
The cache is updated during discovery, but only if it is >22hrs old.
Additionally, the cache is not used and a full yaml load is forced if the cache is >24hrs old.
The cache time is controlled by $config['os_def_cache_time'], I don't see any reason anyone would change that time.

* Delete Session.php

* Include the cache directory.
2017-05-10 08:57:10 +01:00
Neil Lathwood
34c35f3627 webui: Updated remaining display options where we do not show sysName if hostname is IP (#6585)
* webui: Updated remaining display options where we do not show sysName if hostname is IP

* renamed function + simplified
2017-05-05 06:25:58 -05:00
Tony Murray
4dbab407bb refactor: OS discovery tests are now dynamic (#6555)
* refactor: OS discovery tests are now dynamic
All that needs to be done now is supply snmprec files with the proper name. os.snmprec or os_description.snmprec
I also tweaked the loading of yaml files to speed up the tests substantially.
Verifies all os have at least one test and all snmprec files are tested.

* fix typo
2017-05-03 15:52:53 +01:00
Tony Murray
4c0412b14d feature: Wireless Sensors Overhaul (#6471)
* feature: Wireless Sensors
Includes client counts for ios and unifi
Graphing could use some improvement.
Alerting and threshold ui not implemented

WIP: starting OO based wireless sensors.

Class based functionality working
remove old functional files
add schema file

discovery needs to be enabled, not polling

fix up schema

fix Unifi discovery not returning an array

Add some debug when discovering a sensor.
Fix style.

Add missing semicolin

Add a null object (Generic) for OS.
Fill out some phpdocs

Re-organized code
Each sensor type now has it's own discovery and polling interface
Custom polling tested with Unifi CCQ

Left to do:
Implement UI (Graphs and Custom thresholds)
Alerting
Testing

Fix event message text

Remove runDiscovery and runPolling from OS, they are unused and don't belong there.

Cleanups/docs

Missed this file.

Remove the requirement to fetch the current value to check validity.
Do that automatically if current is not specified
A few cleanups here and there

First pass at graphing.
device_ and wireless_ graphs added.

Add RouterOS support

Singleton OS instance isn't required right now.
Remove that to allow some memory to be freed.

Add wireless to the device list metrics.
Make all metrics clickable

Tweak graphs a bit

Implement limit configuration page.
Use sensors page as common code instead of duplicating.
Clean up some javascript interactions:  Allow enter on values to save. Cancel if update is not needed. Enable the clear custom button after setting a custom value.
Add some wireless alert rules to the library.

Add documentation.

Add unifi client counts by ssid in addition to radio.
Optimize Sensor polling a bit.

Add HP MSM clients support (for full controller)
Fix function accessibility

Formalize the discovery and poller interfaces.

Add Xirrus clients and noise floor
move module interfaces to a more appropriate place.
push caching code up to os, unsure about this do to the limitations

No point in selectively enabling wireless discovery.  We only discover if the device supports something.

Add RSSI, Power, and Rate.
Add these sensors for Ubnt Airos.
Clean up some copyrights.

Reduce the amount of files need to add new types.
Leave graph files for consistency and to allow customization.

Remove the old wifi clients graph completely.
ciscowlc should have improved counts (total and per-ssid)

Schema didn't get added.

Impelement the rest of the AirOS sensors
Reformat and re-organize the Airos.php class.

Add several UBNT AirFiber sensors

A few fixes add links to the section headers

Add HP MSM mibs.

* Schema file got dropped in rebase.

* Add wireless menu to view sensors across all devices.
Icons in the menu need help :/

* Add HeliOS, Mimosa, and Siklu support
Sensors added SNR + Noise

* Add power and utilization to Unifi

* Update polling to prefetch all sensor data in a few snmp requests as possible

* Add Extendair: tx+rx power, aggregate rate, frequency

* Add a check for duplicate sensors in discovery.  Just print an error for now.

* Add Bit Error Ratio (named error-ratio to allow for bit error rate to be added if needed)
Fix an incorrect link in the wireless sensors table

* Add error rate and change all bps and Hz to use si units

* Fixes to limits and frequency display

* Fix overview graph frequency display
A few decimal place tweaks

* Don't allow switching sensor and wireless-sensor graphs, it doesn't work.
Change individual distance graphs to use si units

* Go through the OS and make sure I got all the sensors I can (probably missed some still)
Because pollWirelessChannelAsFrequency() is generic and a little complex, so pull it up to OS.
Message to help developers adding supports that don't return an array from discover functions.

* Fix some issues

* Remove noise and signal for now at least
A couple more fixes
Add a notification

* Oopsie

* Bonus AirFiber sensors
2017-05-01 23:49:11 -05:00
Neil Lathwood
056ddbac05 webui: Fixed the display date for the current version (#6474)
* webui: Fixed the display date for the current version

* changed to use exec

* changed >> to >
2017-04-22 18:03:14 -05:00
Tony Murray
72ce39cf9b refactor: schema updates (#6370)
Test the schema files with STRICT_TRANS_TABLES and fix any issues
2017-04-13 10:18:12 +01:00
Neil Lathwood
1bbbaff58b refactor: Moved ifLabel -> cleanPort and updated the usage (#6288)
* refactor: Moved ifLabel -> cleanPort and updated the usage

* added list of tags allowed

* few updates as per comments from @murrant

* Added simple unit tests

* double display for javascript popover

* Remove ifNameDescr() function
Fix realtime port page

* remove accidental temp code

* Remove double display calls in ajax tables
2017-04-04 08:08:23 +01:00
Tony Murray
4b9f3f37d7 fix: move user preferences dashboard and twofactor out of users table (#6286)
* fix: move user preferences dashboard and twofactor out of users table
This allows them to work with any authentication method
Add set_user_pref() and get_user_pref() helper functions

* fix edit users for other users

* Fix updated_at default timestamp

* Update and rename 183.sql to 184.sql

* removed commented out debug
2017-04-01 22:18:00 +01:00
Aldemir Akpinar
2dd0ab870f Do not show unintended debug messages (#6296)
* Do not show unintended debug messages

* Fix the silly mistake
2017-03-29 14:35:44 +01:00
Tony Murray
1ea7af424b feature: bind user for active_directory auth (#6255)
* feature: bind user for active_directory auth
Optional, allows the use of "remember me", API, and alerting.

* missing global (but still may not be working)

* always return a value from reauthenticate()

* Make sure the ldapbind credentials are correct on reauth.
Do not send output if they are incorrect (use d_echo) this breaks ajax calls, etc.
Add scripts/auth_test.php, to make it easier to debug authentication.

* Refine auth_test.php a bit more
A few small cleanups in other places of the auth

* Add auth_test.php to docs
Some more improvements in the auth_test.php output.

* Update Authentication.md
2017-03-29 08:22:02 -05:00
Neil Lathwood
d00a2d6215 fix: Updated set_null() function to be more useful #6030 (#6049)
* fix: Updated set_null() function to be more useful #6030

* drop isset

* Added unit test

* change to assertNull
2017-03-07 22:15:22 -06:00
Neil Lathwood
ed02e39347 webui: Update device overview to not show hostname when certain conditions match #5984 (#6091) 2017-03-04 21:25:06 -06:00
Neil Lathwood
46198b2caa feature: Added ability to show links to fixes for validate (#6054)
* feature: Added ability to show links to fixes for validate

* Finish this PR up and add a few more messages
Move collation checks to print_list()
2017-03-02 08:04:29 +00:00
Richard Mayhew
5bfd23e7eb webui: New eventlog severity classification (#5830)
* New event log severity model

* Revert "New event log severity model"

This reverts commit 0783527af8.

* New event log severity model

* Fixed spaces vs tabs

* Learning the format requirements

* Learning the format requirements

* Moved the colours in to the stylesheet

* change SQL file to stop conflicting

* refactor log_event so severity isn't last, but reference is.

* keep having to move the file due to new PR conflicts

* Revert "keep having to move the file due to new PR conflicts"

This reverts commit f3030e7218.

* extended support to dashboard and eventlog page

* formatting
2017-02-12 22:41:05 +00:00
Søren Rosiak
414efb3539 refactor: Various Code Cleanup (#5777)
* refactor: Various Code Cleanup

* moar
2017-02-05 22:37:08 +00:00
Neil Lathwood
a19b2614dc fix: added some timeout safeguards for curl calls (#5743) 2017-02-03 08:12:42 -06:00
Neil Lathwood
981e974a70 fix: Check lat/lng are numeric rather than !empty #5585 (#5657) 2017-01-31 08:07:27 +00:00
Neil Lathwood
483c40eddd api: Added parent_id to VMs that have a host #3523 (#5621) 2017-01-26 22:38:43 +00:00
Tony Murray
2f1095cb8d fix: html purify init wasn't done always when it was used (#5626) 2017-01-26 19:23:03 +00:00