*  Adds koanf support

* 

 Fix config order

Co-authored-by: jkrvivian <jkrvivian@gmail.com>

* adds default entry node

* updates default cfg values

* refactor plugins

* fix fmt

* fix compile err

* fix errors

* expose singletons via sync.once

* re-disable remotelog and webauth plugins

* re-enable autopeering plugin

* Apply suggestions from code review

* Apply suggestions from code review

* add descriptive comments for getters

Co-authored-by: Luca Moser <moser.luca@gmail.com>

Adds new analysis-dashboard, improves analysis server, adds proper metrics collection and Prometheus exporter (#502)

*  Clean up

*  Add TestCalculateBranchOfTransaction

*  Upgrade fpc dashboard

*  Fix docker-compose

*  Add TestMoveTransactionToBranch

*  Add TestFork

*  Add TestBookPayload

* Add test for checkPayloadSolidity

*  Add TestSetTransactionPreferred

* Add more tests

*  Fix Tangle test

* Feat: started implementing lucas test cases

* Feat: fixed some issued + further tests

* Feat: started adding invalid txs check

* Feat: added removal logic for invalid transactions

* Refactor: removed Println

*  Add test for 2nd Reattachment

* feat: Add first value transfer integration test

* fix: fix wrong plugin name

*  Add aggregated branches test cases

* Feat: added a method to generate AggregatedBranchIDs

*  Use GenerateAggregatedBranchID in test

* Feat: refactored delete logic

* Fix: fixed broken test

* Feat: added final test cases for invalid txs / payloads

* ja nei was gaht dän da ab

*  Make FPCHeartbeat smaller

*  Split vote context to fit into an FPC update

*  WIP

* Value tangle concurrency tests (#451)

* Add simple concurrency test

* Add reverse and concurrent transaction and value object solidification tests and fix bug when value object was visited more than once

* Add some documentation to make tests easily understandable

*  Add FPC analysis data persistence

*  Add Prometheus

*  Update FPC test

*  Remove linter warning

*  Remove linter warnings

*  Fix event closure

*  Add prometheus config

* WIP propagation tests but fixed already couple of bugs

* Fix: fixed some bugs

* Feat: added propagation to inclusion states to tx and its outputs

* Feat: finished the propagation down to the tx and its outputs

*  WIP

*  update docker-network entry node

*  Add .gitignore

*  Update Prometheus

*  Fix MongoDB ctx bug

* WIP propagation tests and fix bugs

*  Delete finalized conflicts

* Add colored tokens test

* Add value tangle test to github workflow

* fix: Fix wrong function name in comments

* refactor: Make testSnapshots disabled in default and minor tweaks

* Feat: fixed some issues and introduced a Debugger

* Refactor: added a few comments

* Split massive test file into slightly more digestible chunks

* Initial commit for metrics collections package

* Clean up propagation tests

* Feat: fixed bugs

* Feat: enabled missing tests

* Add some documentation and missing checks for aggregated branches

*  WIP

* Clean up tangle tests

* adds snapshot type

* Fix: finalized wasn't propagated when a branch was rejected

* Skeleton for implementation

* Measure isSynced

* UI Improvements

* Make it compile

* Measure TPS in value tangle

* implements ReadFrom and WriteTo for Snapshot

* read in snapshot file if snapshot path is defined

* renames snapshot test file

*  WIP metrics

* Measure Tips in value tangle

*  Add DBSize metric

* WIP debugging concurrency bug of death

* Measure tips in message tangle

*  Add AvgNeighborConnectionLifeTime metric

*  Bump hive.go

*  Add autopeering distance metric

* Feat: added more reliable fails in test case

* Fix: fixes a race condition in solidification

*  Add gossip network traffic metric

* Clean up test

* adds assets volume to integration test containers

* fixes some asserts

* adds non-working conflict integration test

* check transaction availability in partition

* renames integration test

* Measure MPS per payload type

* Measure (cumulative) total msg count and per payload

* Measure FPC number of currently active conflicts

* lower amount of peers

* Package updates

*  Link valuetransfer+FPC+dashboard

*  Update packr

*  Fix logo

* first passing version of consensus integration test

* remove debug printlns

* do all integration tests again

* increases avg. network delay fcob rule, removes debug printlns

* go mod tidy by Marie Kondō

* renames incl. state. conflict to conflicting

* go fmt tangle.go

* go fmt tangle_test, goimports dapp.go

* goimports again because the dog is sad

* run consensus integration test on the CI

* use explicit pumba version 0.7.2

* pray to the CI gods for the test to pass

* fix panic when tangle.Fork() is called

*  Add double spend test

*  Update hive.go

* readd all tests again

*  Add AnalysisOutboundBytes metric

*  Update go.mod

*  Start setting prometheus metrics

* reset integration framework paras

*  Update docker-compose

*  Fix typo

*  WIP

*  Update docker-compose

*  Bump golang version

*  Fix snapshot file print

*  Change client url for testing

*  Remove pflag double import

* OpinionEvent struct for calling Finalized and Failed vote events

* Measure finalized/failed conflicts + average rounds of finalization

* refactor: Remove FPC page from dashboard

* refactor: Remove Drng link from dashboard

*  Add autopeering network traffic metric

*  Bump up golang version

*  WIP

* Measure voting queries (received, unreplied) and number of opinions

*  Add sendPayload API

*  Avoid sending empty rounds

*  Add a bunch of metrics to prometheus

* Feat: outputs inherit status of transaction

*  Add Grafana integration

* Refactor: fixed erroneous rename

*  WIP

* Fix: fixed missing marshaling of output bools

* Fixes after merge

*  Fix some bugs

*  Prometheus FPC data collection

*  Small fixes

* Fix: fixed decision pending

*  Prometheus Tangle metrics data collection

*  Refactor FPC metric events

*  Bump up hive.go

*  Add metric heartbeat packet

*  Add clients metrics collection via the analysis server

*  WIP

* Prometheus clients info

* Fix metrics config + enable prometheus collection on entry_node

* use new protocol

* wip: redial on lost connection

* assure conn is not nil

* close on write

* Fix bugs + enable spammer on peer_master

* Fix analysis fpc livefeed bug

* connector cleanup

*  Autopeering NeighborCount + Network Diameter metrics

* do not log the dial error since the logger might not available

* removes test snapshot plugin

*  Improve analysis-server

*  Change log to debug level

* graph pkg from autopeering-sim

* Remove debug line + hive.go update

*  Change hive.go version

* get rid of test snapshot plugin

* fixes wrong use of Println

* removes random tool

*  Move conn initialization

* removes duplicated value entry in GH CI workflows

*  Update hive.go

*  Clean go.mod

*  Fix graph pkg linter warnings

*  improve analysis plugin

*  Fix metrics plugin linter warning

* xxx

* wip

* fixes integration test

*  Add FPC global metrics to prometheus

*  Fix double register

*  Fix client pkg linter warning

*  Fix webapi linter warnings

*  Run you fools!

*  Fix testutil import

* Adjust inbox worker pool capacity to default

* Adjust inbox worker pool capacity to default (#505)

*  Address PR review comments

*  Fix watch dog warnings

*  Let's try with a new bone for the dog now..

* upgrade hive.go to master

*  Fix local dashboard

*  Improves mongoDB reliabilty

*  Fix linter warnings

*  Fix test

*  Make dbSize a prometheus Gauge

* 

 Remove unecessary initialization

* introduce worker pool for storing finalized vote ctxs

* Update packr

* type alias FPCRecords to []FPCRecord

Co-authored-by: jonastheis <mail@jonastheis.de>
Co-authored-by: Hans Moog <hm@mkjc.net>
Co-authored-by: jkrvivian <jkrvivian@gmail.com>
Co-authored-by: Luca Moser <moser.luca@gmail.com>
Co-authored-by: Levente Pap <levente.pap@iota.org>
Co-authored-by: Martyn Janes <martyn@obany.com>
Co-authored-by: Wolfgang Welz <welzwo@gmail.com>

*  Add panic if backgroundWorker fails

*  Use log.Panicf instead of panic

* normalize bind addresses prints

* refactors analysis plugins

* adds tests for parsing/serializing heartbeat packets

* adds additional test case

* normalize usage of word 'ids'

* updates packr files

* normalizes handler routes for static assets for the analysis web interface

* normalize config key names

* adjusts configs for new analysis config keys

* fixes review comments

* adapt to hive.go autopeering events changes

* add exposed function to start autopeering neighbor selection

* start autopeering neighbor selection when gossip is started

* update go modules

* Apply suggestions from code review

Co-Authored-By: Angelo Capossele <angelocapossele@gmail.com>

* fix build errors

Co-authored-by: Angelo Capossele <angelocapossele@gmail.com>

* Update analysis/client with Heartbeat message type

* Refactor analysis/server for heartbeat messages

* Bug fix in analysis server

* Fix heartbeat packet marshaling

* Fix `nodes` and `links` maps handling

* Fix in cleanup in recordedevents

* Fix typos

* Fix event log and heartbeat handling

* Refactor analysis server

 * Remove obsolete packet types
 * Run Analysis Server Record Manager in background
 * Refactor Analysis Client with ticker

* Define max amount of neighbors to report

* Small fixes

* New visualization for analysis server

* PR review fixes

* Remove `nodeOnline` and `nodeOffline` events from analysis server

* Refactor analysis frontend

* Fix bug in websocket, remove obsolete triggers

* Remove debug vars, re-enable websocket ping

* Stop ws pings when closed

* Color settings for visualizer

* Update graph colors

* Fix no neighbor bug

* Dynamically change springLength based on nodes

* Small color fix

* Rename ShutdownPriorityAnalysis

* Formatting, linter suggestions

* Update packr file for visualizer

* Websocket Write() instead of TryWrite()

* Feat: started to merge changes

* Refactor: moved parameter package to be a plugin (same with logger)

* Feat: first compiling version of new ontologoies merge

* Feat: ported additional plugins

* Feat: transaction get solid now

* Refactor: reverted some previous changes from debugging

* Feat: added a banner module for the cli interface

* Feat: added a plugin for the port checks

* Feat: fixed some bugs

* Refactor: reverted some changes

* Feat: reworked TransactionParser to use Errors

* Feat: TransactionParser uses Peer

* Feat: started to rework broadCastData webapi call

* Feat: refactored some plugins

* Fix: fixed test of tangle

* Refactor: changed tangle package in graph plugin

* Refactor: uncommented broken code

* Fix: fixed broken method signature in gossip test

* Feat: started adding value tangle

* Feat: adjusted to new hive.go

* Feat: upgraded hive.go

* Clean up PortCheck plugin and make it standalone #259 (#271)

Co-authored-by: Hans Moog <hm@mkjc.net>

Co-authored-by: Jonas Theis <mail@jonastheis.de>

* New binary tangle using atomic transactions (#239)

* Feat: started porting the new binary stuff

* Refactor: removed unnecessary folder

* Refactor: cleaned up go.mod files

* Fix: removed objectsdb files

* Feat: added transactionrequester package

* Adds the transactionParser as a fliter and pre-processing mechanism of the tangle. (#242)

* Feat: started porting the new binary stuff

* Refactor: removed unnecessary folder

* Refactor: cleaned up go.mod files

* Fix: removed objectsdb files

* Feat: added transactionrequester package

* Feat: added new transactionparser as the filter for the tangle

* Feat: Use hive.go autopeering (#250)

* use autopeering from hive.go
* update hive.go

* update hive.go

* Adds the TransactionRequester and some refactors (#256)

* Feat: started porting the new binary stuff

* Refactor: removed unnecessary folder

* Refactor: cleaned up go.mod files

* Fix: removed objectsdb files

* Feat: added transactionrequester package

* Feat: added new transactionparser as the filter for the tangle

* Refactor: removed duplicate code

* Fix: Log dropping packets every 1000 drops (#255)

* Fix: Log dropping packets every 1000 drops

* use an atomic counter for dropped message because Write() could be called concurrently

* removes redundant zero

* Add external check in test

Co-authored-by: Luca Moser <moser.luca@gmail.com>

* Fix: Update pprof port (#244)

Since both Hornet and Goshimmer might be running on the same machine, update the port so that we can get debug information for both.

* Remove docker specific config (#245)

*  upgrades hive.go

*  checks meta_tx size

*  removes dependencies

*  enables local testing

*  adds config plugin

* Extend remote log message (#260)

* Bump up Logstash and ElasticSearch memory #251

* Add version and if available GIT information to remote log message #251

* 

 fixes flag parsing before loading config file

* Feat: Add --version to cli (#264)

* Adds changelog entry for v0.1.2 and bumps version number (#265)

* updates changelog for v0.1.2 release

* bumps version number to v0.1.2

Co-authored-by: Hans Moog <hm@mkjc.net>
Co-authored-by: Wolfgang Welz <welzwo@gmail.com>
Co-authored-by: jkrvivian <jkrvivian@gmail.com>
Co-authored-by: Dave <44786846+centercirclesolutions@users.noreply.github.com>
Co-authored-by: Angelo Capossele <angelocapossele@gmail.com>
Co-authored-by: Jonas Theis <mail@jonastheis.de>

* use autopeering from hive.go
* update hive.go

* fix: ignore rejected peering requests

* Do not store local services in DB

Signed-off-by: Luca Moser <moser.luca@gmail.com>

*  GetNextCandidate selection added

*  Recompile salt.proto

*  Refactor peer IDs completely

*  Recompile proto sources

*  WIP

*  renamed to sort

*  Return requested peers on request

*  Kepp track how ofter a managed peer was verified

*  sort adapted to the new peer

*  Query verified peers for new peers

*  WIP

*  Add manager tests

*  Add peering messages to protobuf

*  Make Salt methods consistent with Peer

*  Handle peering messages

*  WIP

*  Add To filed to PeersRequest

*  Add PeersRequest test

*  Adding PeersRequest benchmark

*  Move mpeer and helper functions to own file

*  Bump peer only once on reverification

*  WIP

* 

*  WIP

*  Increase buffer to prevent deadlocks

*  Remove unnecessary lock

*  Add peering request test

*  Make tests deterministic by triggering reverify

*  WIP

*  fixing concurrency issues

*  testManager improved

*  Don't send PeeringResponse for invalid salt

*  WIP

*  WIP

*  removed commented code

*  WIP

*  WIP

* Neighbourhood manager (#11)

*  GetNextCandidate selection added

*  fixing concurrency issues

*  testManager improved

*  removed commented code

*  WIP

*  WIP

*  WIP

*  WIP

*  Make TestAdd pass in neighborhood_test

`Add` should do nothing if the neighborhood list is full.

*  Improve TestSrvVerifyBoot

*  Upgrade to go 1.13

*  Use testify instead of magiconair/properties

* 

*  WIP

*  WIP

*  WIP

*  WIP

* Simulation (#14)

*  clear rejection filter after refreshing the public salt

*  clean rejection filter

*  Add mpeer test

*  gofmt test

*  Remove ineffactual assignments

*  Trigger panic when index is out of bound in deletePeer

*  Add selection test

*  Add TestGetFurtherest and TestGetPeerIndex

*  both salt updated at the same time and neighborhood dropped

*  Add node peering history for simulation

*  WIP

*  added sim visualization

*  removed root handler

*  added README

*  updated README

*  removed unused dependencies

*  Tidy go.mod

*  Work in progress

*  WIP

*  WIP

*  WIP

*  improved start button

*  WIP

*  added keyboard support for start

*  WIP

*  WIP

*  WIP

*  WIP

*  WIP

*  WIP

*  added input parameters

*  README updated

*  gif updated

*  figure updated

*  updated

*  updated gif

* removed simulation metrics

*  updated

*  updated

*  Extract server from discovery and peering

*  Use golang/protobuf

* Update README.md

*  Rename PeersRequest to DiscoveryRequest

*  Fixing typos

*  Move unused function to tests

*  The selection protocol depends on the discovery

*  Make tests more robust when using -race

*  Improve logging

*  Remove unnecessary arguments

*  Fix data races

*  added timeout for simulated network

*  added loop to animation

*  rename neighborhood to selection

*  adds initial salt and fixes simulation end

*  visualizer enabled by default

*  new parameter dropAll and improved python script

*  updated README

*  updated README

*  fix salt initialization

*  added blogpost link

*  Add badges to README

*  Add Travis CI for tests

*  Correct formating

*  Running golangci on Travis

*  Ignore return value of 'logger.Sync'

*  Remove unused functions

*  Add link to license

*  Move simnetwork to transport

*  Use the complete network protocol in simulations

*  Do not export selection/manager

*  Remove gRPC transport layer

*  Add UDP connection transport test

*  Implement the peer DB using Goshimmer database

*  Use the local GoShimmer version

*  Add support for a persistent database

*  Persist private key of local peer in database

*  Set TTL for bootstrap peers

*  Use GoShimmer events

*  Store the supported services in the local peer

*  Use most current GoShimmer git version as a dep

*  Switch to hive.go event package

*  Use correct module iotaledger/autopeering-sim

*  Provide dummy service in autopeering simulation

*  adds service support in the selection

*  adds peer discovered events

*  adds GetIncomingNeighbors and GetOutgoingNeighbors

*  fixes out of bound error in splitNodeKey

*  adds public IP support

*  fixes localhost

*  fixes localhost parsing

*  changes selection loop to 1s

*  switches from fmt.Println to log.Debug

*  increases maxKnown to 1000 and always triggers discovered peer

*  adds PeerDeleted event

*  moves PeerDeleted event into deletePeer

*  adds config paramters for the peer discovery

*  adds config parameters to neighbor selection

*  enable/disable inbound/outbound selection

*  Improve Godoc comments

*  modifies disabled outbound selection

*  fixes bug with disabling the selection

*  removes getMyIP() from server

*  removes some debugging logs

*  Introduce services

- Each peer now comes with a set of services
- Local peer is a proper peer
- Services are exchanged during Pong and are available for all verified
peers

*  fixes GetVerifiedPeer

*  adds gossip key to the service

*  removes debugging logs

*  Add test for GetVerifiedPeer

*  Fix main

*  Add localNetwork to Protocol

*  Add new but verified peers to the manager

*  changes configurable parameters

*  fixes DiscoveryRequest field sequence

*  Regenerate proto files

*  Cleanup parameters

*  Fix potential race condition

*  Reduce logging verbosity

*  Add test for selection+discover

*  Return net.Addr in Transport

*  Remove inbound/outbound switches completely

*  Improve logging

*  Fix peerdb logs

*  Fix peer updating

*  Make TestProtFull more robust

*  Make queryInterval a parameter

*  Improve loggind during querying

*  Trigger PeerDiscovered only for active peers

*  Cleanup protocol tests

*  Add discovery test on protocol level

*  Rename maxVerified to maxManaged

*  Increase default query interval

*  Improve discover benchmarks

*  Fix manager tests

*  Do not use bugged assert.Eventually

*  Fix linter warnings in server

*  Remove unused parameters

*  Make transport work on slices of bytes

*  Fix typo in comments

*  Fix linter warnings

*  UpdateService accepts two strings

*  Add test that services are received in discover

*  adds required services

*  Handle closed connections consistently

*  Code cleanup

*  fixes DropPeer

*  improves debug messages

*  Log errors during reverification

* 

 Log packet size

* refactor: remove unused files

* refactor: use internal autopeering package

Co-authored-by: Angelo Capossele <angelocapossele@gmail.com>
Co-authored-by: jkrvivian <jkrvivian@gmail.com>