Code

Threat detection#1783

Code

Draft

ag060 wants to merge 89 commits intomasterakto-api-security/akto:masterfrom threat-detection-v1akto-api-security/akto:threat-detection-v1Copy head branch name to clipboard

+9,854-1,468

Commits

Commits on Jan 8, 2025

• pushing malicious requests to kafka

  ag060

  committed
  09d279c

• added flush kafka messages task

  ag060

  committed
  a1c1f47

• added window based aggregation support for malicious requests

  ag060

  committed
  a450ed1

• using separate mongo for storing suspect data

  ag060

  committed
  b17132f

• fixed kafka host

  ag060

  committed
  2dcb947

• fixed source ip generation logic

  ag060

  committed
  786df40

• not syncing writes and read for RedisWriteBackCache at the same time

  ag060

  committed
  a8445b3

• using redis backed increment counter cache

  ag060

  committed
  ec21b33

• added group id to aggregate notifier

  ag060

  committed
  dedd012

• added notification cooldown support

  ag060

  committed
  fa47ad7

• added start and end bucket ids with each detected alerts

  ag060

  committed
  14ca6ec

• added ttl for redis entries

  ag060

  committed
  de83383

• added new collections for sample malicious requests and detected alerts

  ag060

  committed
  64c20b7

• addded cleanup sample malicious records task

  ag060

  committed
  604cbae

• refactor code

  ag060

  committed
  3dd5ae8

• refactor code

  ag060

  committed
  121429e

• refactor code

  ag060

  committed
  4d6c56e

• add aggregation parse layer

  

  ayushaga14

  authored and

  ag060

  committed
  71413e3

• grpc service for consuming malicious and smart events (#1748)

  Show description for edad02a
  ag060

  committed
  edad02a

• apply aggregations

  

  ayushaga14

  authored and

  ag060

  committed
  821581c

• removed db service and using mongo client everywhere

  ag060

  committed
  aa5fa0c

• moved clean up task to threat protection service

  ag060

  committed
  6972092

• added authorization interceptor for grpc server

  ag060

  committed
  6ea40da

• removed account_id field from consume event service requests (threat-detection-backend)

  Show description for 32f1abe
  ag060

  committed
  32f1abe

• added client side authentication support while saving malicious and smart events

  ag060

  committed
  88367f6

• added new api threat detection

  Show description for f0c5513
  ag060

  committed
  f0c5513

• saving malicious sample data to postgres db

  ag060

  committed
  e877ad2

• abstracted out common kafka polling consumer task

  ag060

  committed
  2e13707

• added tasks for sending alerts to protection backend

  ag060

  committed
  1de7a3d

• exposing single rpc method for recording alerts

  ag060

  committed
  7f4d14a

• updated proto definition

  ag060

  committed
  151eab0

• removed cleanup task from threat protection backend module

  ag060

  committed
  69fa966

• updated kafka topic names

  ag060

  committed
  f184d0a

• running flyway migrations programatically

  ag060

  committed
  e088a38

• refactored code

  ag060

  committed
  5ba40d4

• added clean up for deleting all the malicious entries older than 7 days

  ag060

  committed
  9c701c7

• added hibernate orm (#1786)

  ag060

  committed
  c01d933

• threat protection producer consumer db writes

  

  ayushaga14

  authored and

  ag060

  committed
  3435917

• fix create mongo client params

  

  ayushaga14

  authored and

  ag060

  committed
  7e13496

• deleted generated proto files

  ag060

  committed
  4ba5e5f

• added dashboard rpc service (#1808)

  ag060

  committed
  f2cb97c

• updated proto definition to include malicious event type (SINGLE or AGGREGATED)

  ag060

  committed
  9ab05f4

• storing api_collection_id in postgres

  ag060

  committed
  38037f6

• fixed paths in ci for threat-detection module

  ag060

  committed
  475c871

• moved kafka config from threat detection to utils

  ag060

  committed
  2d8b80f

• moved consumer and producer on same machine for threat protection backend

  ag060

  committed
  bc5a8e9

• refactored dashboard proto messages

  ag060

  committed
  4955950

• fixed MaliciousEventModel not populating data from mongo

  ag060

  committed
  6838d40

• refactored code

  ag060

  committed
  8311596

• disabled debug logging for hibernate

  ag060

  committed
  5f99aea

• removed unused imports

  ag060

  committed
  2c5fe25

• disabled debug logging for hibernate

  ag060

  committed
  c48f9cd

• committing kafka offsets manually now

  ag060

  committed
  58cf88b

• renamed threat detection backend service

  ag060

  committed
  c765f77

• added health check server

  ag060

  committed
  a496b43

• building threat detection backend image in staging workflow

  ag060

  committed
  6c474f7

• refactored generated proto files

  ag060

  committed
  f3442ed

• added health check for rpc and enabled reflection

  ag060

  committed
  77cae61

• added health service for grpc

  ag060

  committed
  789ff5e

• using vertx http server as threat detection backend (#1842)

  ag060

  committed
  b0f5320

• reading akto threat detection backend url from env

  ag060

  committed
  0bdf17d

• added try catch

  ag060

  committed
  bea57d3

• fixed sample not being to backend

  ag060

  committed
  586b1ae

• clearing cache as soon the alert is raised

  ag060

  committed
  f780063

• sending sample malicious events to backend only once

  ag060

  committed
  9372bbc

• use fastjson lib and increase template fetch interval

  

  ayushaga14

  authored and

  ag060

  committed
  b3a17f6

• add log

  

  ayushaga14

  authored and

  ag060

  committed
  c810c5a

• avoid reevaluating params for each filter check

  

  ayushaga14

  authored and

  ag060

  committed
  4ba026e

• [temp commit] remove unneceesary build from stagiong workflow

  

  ayushaga14

  authored and

  ag060

  committed
  59c70a2

• using proto message envelope for kafka transport in threat detection client (#1874)

  ag060

  committed
  3439bca

• setting redis ttl while sync to redis instead of increment op

  ag060

  committed
  779e3e1

• fixed merge conflicts in staging and prod workflows

  ag060

  committed
  f0314f9

• proto changes

  

  ayushaga14

  authored and

  ag060

  committed
  65d63d8

• remove logs and headers loop

  

  ayushaga14

  authored and

  ag060

  committed
  8139956

• optimizations

  

  notshivansh

  authored and

  ag060

  committed
  a145b9f

• move httpresponse param proto generated files

  

  ayushaga14

  authored and

  ag060

  committed
  938f4c1

• fixed bad merges

  ag060

  committed
  3220902

• refactored threat detection backend

  ag060

  committed
  5eaa0b3

• maxmind integration (#1924)

  ag060

  committed
  44c48e1

• added api protection pages in left nav

  ag060

  committed
  52b70e5

• removed System.out.println

  ag060

  committed
  d8a1470

• removed unusued proto messages

  ag060

  committed
  489fb59

• reading kafka key and value serializer from config

  ag060

  committed
  dd567f0

• updated custom generated httpresponse params files

  ag060

  committed
  2496f82

Commits on Jan 9, 2025

• moved threat policies to separate page

  ag060

  committed
  20a4cdd

• fixed incorrect total count of entries in final result for list threat apis

  ag060

  committed
  44c632a

• fixed sorting in threat detection tables

  ag060

  committed
  f2cc035

• fixed prettify endpoint url if url is empty (home page)

  ag060

  committed
  3be068a

Commits on Jan 10, 2025

• added filters support in threat detection APIs

  ag060

  committed
  926bdf6