librenms/app/Models/Mempool.php

186 lines
5.1 KiB
PHP
Raw Normal View History

<?php
namespace App\Models;
Modernize mempools (#12282) * mempools to modern module quick hacky hrstorage port * port ucd-snmp-mib to Mempools * Populate DB for ucd Prep for yaml * initial yaml attempt * more complex conversions fixes to YamlDiscovery, make leading $ optional and allow mib::oid format * walk full tables and skip values normalize percentages above 100 * handle precent only ones (specify total as 100) * Move default polling out of YamlMempoolsDiscovery * fixes * Update test data hrstorage should be correct. * perc_warn for hrstorage * Host Resources, record buffer, cached, and shared * Host Resources is always better, don't do both HR and UCD * fix unix, include warning levels * variable size * consolidate skip_values * define mempools schema * number instead of integer * more schema refactor * one more skip_values reference * throw error for invalid oid translation aos6 * a* and Cisco * updated test data * update almost all hrstorage data files * b* * c* with test data use standard cache for hrStorage * use cache for storage module too * hand bsnmp properly * bdcom * exclude total oid from yaml so it is not polled May add a way to ignore this behavior and poll it, but I don't know if that is needed. * automatically handle percent only values * ciscowlc * only poll used or free if we have used, free, and total. * fix skipping * the dlinkoning fix find value when value "name" is numeric * support numeric oids * dnos/ftos attempt * I guess we can't filter on total > 0 * edgecos * e* * f WIP * f* * gwd (aka g*) * h* + procurve * i* * j* * m* * support 0% used memory (however unlikely) * n* * CISCO-PROCESS-MIB memory, share cache with processors module * ignore mempools with invalid total * p* * quanta * r* fix raisecom mibs terribly broken * s-z * style fixes * Move VRP back to PHP and make it actually work * fix zynos * update schema * Update Cisco processor data for description bug fixes * fix comware processors * comware mempools with memory size default precision to 1 * sophos-xg updated data * hrstorage use ram size for buffers, cache, and shared * Show memory available instead of free in device overview * UCD, use same rrd format, store available instead of free in the db. * Calculate availability for HOST-RESOURCES-MIB * Convert UCD to standard polling * rename old rrd files * initial graph work * graph WIP * Graph looking decent * Graph looking decent for hr * remove old ucd_graph code * handle availability mempool more graph cleanup * color adjustments * remove accidental free calculation * Update test data and fix corner cases * fis pfsense * update schema * add default value for mempool_class * fix whitespace * update schema * update schema correctly * one more time * fortigate_1500d-sensors missing oids * Update docs. * fix indent * add implements MempoolsDiscovery explicitly to OS * remove ucd_memory graph references remove unused device_memory graph * remove unused functions * set devices with mempools to rediscover to prevent/minimize gaps * use a subquery * add overview graph * port health mempools table * Update device mempool * only show overview if multiple * Don't override user set warn percentages in discovery * fix missed usage * fix style * Safety check to not rename rrd files incorrectly if migration has not been run. * Fix overview percent bar and represent available and used on the bar * missed an item to convert to mempool_class * percent on the wrong side
2020-11-23 21:35:35 +00:00
use Illuminate\Support\Str;
use LibreNMS\Interfaces\Models\Keyable;
class Mempool extends DeviceRelatedModel implements Keyable
{
protected $table = 'mempools';
protected $primaryKey = 'mempool_id';
Modernize mempools (#12282) * mempools to modern module quick hacky hrstorage port * port ucd-snmp-mib to Mempools * Populate DB for ucd Prep for yaml * initial yaml attempt * more complex conversions fixes to YamlDiscovery, make leading $ optional and allow mib::oid format * walk full tables and skip values normalize percentages above 100 * handle precent only ones (specify total as 100) * Move default polling out of YamlMempoolsDiscovery * fixes * Update test data hrstorage should be correct. * perc_warn for hrstorage * Host Resources, record buffer, cached, and shared * Host Resources is always better, don't do both HR and UCD * fix unix, include warning levels * variable size * consolidate skip_values * define mempools schema * number instead of integer * more schema refactor * one more skip_values reference * throw error for invalid oid translation aos6 * a* and Cisco * updated test data * update almost all hrstorage data files * b* * c* with test data use standard cache for hrStorage * use cache for storage module too * hand bsnmp properly * bdcom * exclude total oid from yaml so it is not polled May add a way to ignore this behavior and poll it, but I don't know if that is needed. * automatically handle percent only values * ciscowlc * only poll used or free if we have used, free, and total. * fix skipping * the dlinkoning fix find value when value "name" is numeric * support numeric oids * dnos/ftos attempt * I guess we can't filter on total > 0 * edgecos * e* * f WIP * f* * gwd (aka g*) * h* + procurve * i* * j* * m* * support 0% used memory (however unlikely) * n* * CISCO-PROCESS-MIB memory, share cache with processors module * ignore mempools with invalid total * p* * quanta * r* fix raisecom mibs terribly broken * s-z * style fixes * Move VRP back to PHP and make it actually work * fix zynos * update schema * Update Cisco processor data for description bug fixes * fix comware processors * comware mempools with memory size default precision to 1 * sophos-xg updated data * hrstorage use ram size for buffers, cache, and shared * Show memory available instead of free in device overview * UCD, use same rrd format, store available instead of free in the db. * Calculate availability for HOST-RESOURCES-MIB * Convert UCD to standard polling * rename old rrd files * initial graph work * graph WIP * Graph looking decent * Graph looking decent for hr * remove old ucd_graph code * handle availability mempool more graph cleanup * color adjustments * remove accidental free calculation * Update test data and fix corner cases * fis pfsense * update schema * add default value for mempool_class * fix whitespace * update schema * update schema correctly * one more time * fortigate_1500d-sensors missing oids * Update docs. * fix indent * add implements MempoolsDiscovery explicitly to OS * remove ucd_memory graph references remove unused device_memory graph * remove unused functions * set devices with mempools to rediscover to prevent/minimize gaps * use a subquery * add overview graph * port health mempools table * Update device mempool * only show overview if multiple * Don't override user set warn percentages in discovery * fix missed usage * fix style * Safety check to not rename rrd files incorrectly if migration has not been run. * Fix overview percent bar and represent available and used on the bar * missed an item to convert to mempool_class * percent on the wrong side
2020-11-23 21:35:35 +00:00
public $timestamps = false;
protected $fillable = [
'mempool_perc_warn',
'mempool_index',
'entPhysicalIndex',
'mempool_type',
'mempool_class',
'mempool_precision',
'mempool_descr',
'mempool_perc',
'mempool_perc_oid',
'mempool_used',
'mempool_used_oid',
'mempool_free',
'mempool_free_oid',
'mempool_total',
'mempool_total_oid',
'mempool_largestfree',
'mempool_lowestfree',
];
protected $attributes = [
'mempool_precision' => 1,
];
public function __construct(array $attributes = [])
{
parent::__construct($attributes);
if (! $this->exists) {
// only allow mempool_perc_warn to be filled for new mempools
unset($this->fillable[array_search('mempool_perc_warn', $this->fillable)]);
}
}
public function isValid()
{
return $this->mempool_total > 0;
}
public function fillUsage($used = null, $total = null, $free = null, $percent = null)
{
try {
$total = $this->correctNegative($total);
$used = $this->correctNegative($used, $total);
$free = $this->correctNegative($free, $total);
} catch (\Exception $e) {
d_echo($e->getMessage());
return $this; // unhandled negative
}
$this->mempool_total = $this->calculateTotal($total, $used, $free);
$this->mempool_used = $used * $this->mempool_precision;
$this->mempool_free = $free * $this->mempool_precision;
$percent = $this->normalizePercent($percent); // don't assign to model or it loses precision
$this->mempool_perc = $percent;
if (! $this->mempool_total) {
if (! $percent && $percent !== '0') {
// could not calculate total, can't calculate other values
return $this;
}
$this->mempool_total = 100; // only have percent, mark total as 100
}
if ($used === null) {
$this->mempool_used = $free !== null
? $this->mempool_total - $this->mempool_free
: round($this->mempool_total * ($percent ? ($percent / 100) : 0));
}
if ($free === null) {
$this->mempool_free = $used !== null
? $this->mempool_total - $this->mempool_used
: round($this->mempool_total * ($percent ? (1 - ($percent / 100)) : 1));
}
if ($percent == null) {
$this->mempool_perc = $this->mempool_used / $this->mempool_total * 100;
}
return $this;
}
/**
* Set the mempool class. If no class is given, try to detect it from available data.
*
* @param string $class
* @param string $default
Modernize mempools (#12282) * mempools to modern module quick hacky hrstorage port * port ucd-snmp-mib to Mempools * Populate DB for ucd Prep for yaml * initial yaml attempt * more complex conversions fixes to YamlDiscovery, make leading $ optional and allow mib::oid format * walk full tables and skip values normalize percentages above 100 * handle precent only ones (specify total as 100) * Move default polling out of YamlMempoolsDiscovery * fixes * Update test data hrstorage should be correct. * perc_warn for hrstorage * Host Resources, record buffer, cached, and shared * Host Resources is always better, don't do both HR and UCD * fix unix, include warning levels * variable size * consolidate skip_values * define mempools schema * number instead of integer * more schema refactor * one more skip_values reference * throw error for invalid oid translation aos6 * a* and Cisco * updated test data * update almost all hrstorage data files * b* * c* with test data use standard cache for hrStorage * use cache for storage module too * hand bsnmp properly * bdcom * exclude total oid from yaml so it is not polled May add a way to ignore this behavior and poll it, but I don't know if that is needed. * automatically handle percent only values * ciscowlc * only poll used or free if we have used, free, and total. * fix skipping * the dlinkoning fix find value when value "name" is numeric * support numeric oids * dnos/ftos attempt * I guess we can't filter on total > 0 * edgecos * e* * f WIP * f* * gwd (aka g*) * h* + procurve * i* * j* * m* * support 0% used memory (however unlikely) * n* * CISCO-PROCESS-MIB memory, share cache with processors module * ignore mempools with invalid total * p* * quanta * r* fix raisecom mibs terribly broken * s-z * style fixes * Move VRP back to PHP and make it actually work * fix zynos * update schema * Update Cisco processor data for description bug fixes * fix comware processors * comware mempools with memory size default precision to 1 * sophos-xg updated data * hrstorage use ram size for buffers, cache, and shared * Show memory available instead of free in device overview * UCD, use same rrd format, store available instead of free in the db. * Calculate availability for HOST-RESOURCES-MIB * Convert UCD to standard polling * rename old rrd files * initial graph work * graph WIP * Graph looking decent * Graph looking decent for hr * remove old ucd_graph code * handle availability mempool more graph cleanup * color adjustments * remove accidental free calculation * Update test data and fix corner cases * fis pfsense * update schema * add default value for mempool_class * fix whitespace * update schema * update schema correctly * one more time * fortigate_1500d-sensors missing oids * Update docs. * fix indent * add implements MempoolsDiscovery explicitly to OS * remove ucd_memory graph references remove unused device_memory graph * remove unused functions * set devices with mempools to rediscover to prevent/minimize gaps * use a subquery * add overview graph * port health mempools table * Update device mempool * only show overview if multiple * Don't override user set warn percentages in discovery * fix missed usage * fix style * Safety check to not rename rrd files incorrectly if migration has not been run. * Fix overview percent bar and represent available and used on the bar * missed an item to convert to mempool_class * percent on the wrong side
2020-11-23 21:35:35 +00:00
* @return \App\Models\Mempool
*/
public function setClass($class = null, $default = 'system')
Modernize mempools (#12282) * mempools to modern module quick hacky hrstorage port * port ucd-snmp-mib to Mempools * Populate DB for ucd Prep for yaml * initial yaml attempt * more complex conversions fixes to YamlDiscovery, make leading $ optional and allow mib::oid format * walk full tables and skip values normalize percentages above 100 * handle precent only ones (specify total as 100) * Move default polling out of YamlMempoolsDiscovery * fixes * Update test data hrstorage should be correct. * perc_warn for hrstorage * Host Resources, record buffer, cached, and shared * Host Resources is always better, don't do both HR and UCD * fix unix, include warning levels * variable size * consolidate skip_values * define mempools schema * number instead of integer * more schema refactor * one more skip_values reference * throw error for invalid oid translation aos6 * a* and Cisco * updated test data * update almost all hrstorage data files * b* * c* with test data use standard cache for hrStorage * use cache for storage module too * hand bsnmp properly * bdcom * exclude total oid from yaml so it is not polled May add a way to ignore this behavior and poll it, but I don't know if that is needed. * automatically handle percent only values * ciscowlc * only poll used or free if we have used, free, and total. * fix skipping * the dlinkoning fix find value when value "name" is numeric * support numeric oids * dnos/ftos attempt * I guess we can't filter on total > 0 * edgecos * e* * f WIP * f* * gwd (aka g*) * h* + procurve * i* * j* * m* * support 0% used memory (however unlikely) * n* * CISCO-PROCESS-MIB memory, share cache with processors module * ignore mempools with invalid total * p* * quanta * r* fix raisecom mibs terribly broken * s-z * style fixes * Move VRP back to PHP and make it actually work * fix zynos * update schema * Update Cisco processor data for description bug fixes * fix comware processors * comware mempools with memory size default precision to 1 * sophos-xg updated data * hrstorage use ram size for buffers, cache, and shared * Show memory available instead of free in device overview * UCD, use same rrd format, store available instead of free in the db. * Calculate availability for HOST-RESOURCES-MIB * Convert UCD to standard polling * rename old rrd files * initial graph work * graph WIP * Graph looking decent * Graph looking decent for hr * remove old ucd_graph code * handle availability mempool more graph cleanup * color adjustments * remove accidental free calculation * Update test data and fix corner cases * fis pfsense * update schema * add default value for mempool_class * fix whitespace * update schema * update schema correctly * one more time * fortigate_1500d-sensors missing oids * Update docs. * fix indent * add implements MempoolsDiscovery explicitly to OS * remove ucd_memory graph references remove unused device_memory graph * remove unused functions * set devices with mempools to rediscover to prevent/minimize gaps * use a subquery * add overview graph * port health mempools table * Update device mempool * only show overview if multiple * Don't override user set warn percentages in discovery * fix missed usage * fix style * Safety check to not rename rrd files incorrectly if migration has not been run. * Fix overview percent bar and represent available and used on the bar * missed an item to convert to mempool_class * percent on the wrong side
2020-11-23 21:35:35 +00:00
{
if ($class) {
$this->mempool_class = $class;
return $this;
}
$memoryClasses = [
'virtual' => ['virtual'],
'swap' => ['swap'],
'buffers' => ['buffers'],
'cached' => ['cache'],
'system' => ['shared real memory metrics'],
'shared' => ['shared'],
];
$descr = strtolower($this->mempool_descr);
foreach ($memoryClasses as $class => $search) {
if (Str::contains($descr, $search)) {
$this->mempool_class = $class;
return $this;
}
}
if ($default == 'virtual' && Str::contains($this->mempool_descr, ['/dev/'])) {
$this->mempool_class = 'swap';
return $this;
}
$this->mempool_class = $default;
Modernize mempools (#12282) * mempools to modern module quick hacky hrstorage port * port ucd-snmp-mib to Mempools * Populate DB for ucd Prep for yaml * initial yaml attempt * more complex conversions fixes to YamlDiscovery, make leading $ optional and allow mib::oid format * walk full tables and skip values normalize percentages above 100 * handle precent only ones (specify total as 100) * Move default polling out of YamlMempoolsDiscovery * fixes * Update test data hrstorage should be correct. * perc_warn for hrstorage * Host Resources, record buffer, cached, and shared * Host Resources is always better, don't do both HR and UCD * fix unix, include warning levels * variable size * consolidate skip_values * define mempools schema * number instead of integer * more schema refactor * one more skip_values reference * throw error for invalid oid translation aos6 * a* and Cisco * updated test data * update almost all hrstorage data files * b* * c* with test data use standard cache for hrStorage * use cache for storage module too * hand bsnmp properly * bdcom * exclude total oid from yaml so it is not polled May add a way to ignore this behavior and poll it, but I don't know if that is needed. * automatically handle percent only values * ciscowlc * only poll used or free if we have used, free, and total. * fix skipping * the dlinkoning fix find value when value "name" is numeric * support numeric oids * dnos/ftos attempt * I guess we can't filter on total > 0 * edgecos * e* * f WIP * f* * gwd (aka g*) * h* + procurve * i* * j* * m* * support 0% used memory (however unlikely) * n* * CISCO-PROCESS-MIB memory, share cache with processors module * ignore mempools with invalid total * p* * quanta * r* fix raisecom mibs terribly broken * s-z * style fixes * Move VRP back to PHP and make it actually work * fix zynos * update schema * Update Cisco processor data for description bug fixes * fix comware processors * comware mempools with memory size default precision to 1 * sophos-xg updated data * hrstorage use ram size for buffers, cache, and shared * Show memory available instead of free in device overview * UCD, use same rrd format, store available instead of free in the db. * Calculate availability for HOST-RESOURCES-MIB * Convert UCD to standard polling * rename old rrd files * initial graph work * graph WIP * Graph looking decent * Graph looking decent for hr * remove old ucd_graph code * handle availability mempool more graph cleanup * color adjustments * remove accidental free calculation * Update test data and fix corner cases * fis pfsense * update schema * add default value for mempool_class * fix whitespace * update schema * update schema correctly * one more time * fortigate_1500d-sensors missing oids * Update docs. * fix indent * add implements MempoolsDiscovery explicitly to OS * remove ucd_memory graph references remove unused device_memory graph * remove unused functions * set devices with mempools to rediscover to prevent/minimize gaps * use a subquery * add overview graph * port health mempools table * Update device mempool * only show overview if multiple * Don't override user set warn percentages in discovery * fix missed usage * fix style * Safety check to not rename rrd files incorrectly if migration has not been run. * Fix overview percent bar and represent available and used on the bar * missed an item to convert to mempool_class * percent on the wrong side
2020-11-23 21:35:35 +00:00
return $this;
}
public function setMempoolPercAttribute($percent)
{
$this->attributes['mempool_perc'] = round($percent);
}
public function getCompositeKey()
{
return "$this->mempool_type-$this->mempool_index";
}
private function correctNegative($value, $max = null)
{
$int_max = 4294967296;
if ($value < 0) {
// assume unsigned/signed issue
$value = $int_max + $value;
if (($max && $value > $max) || $value > $int_max) {
throw new \Exception('Uncorrectable negative value');
}
}
return $value;
}
private function calculateTotal($total, $used, $free)
{
if ($total !== null) {
return $total * $this->mempool_precision;
}
if ($used !== null && $free !== null) {
return ($used + $free) * $this->mempool_precision;
}
return $this->mempool_total; // don't change the value it may have been set in discovery
}
private function normalizePercent($percent)
{
while ($percent > 100) {
$percent = $percent / 10;
}
return $percent;
}
}