Hi there. In this article we'll discuss about logstash's installation procedure.  Before technical issues let's speak about our goals and strictly about logstash.   Image situation when you have web resource which work in production, it handles a huge amount of users. Let's say that there are thousands of users.... Each user has his own id. He could add/delete/update something via this web resource and you should know what exactly add or remove each user in some exactly time. Your web resource logs everything to syslog, for example and you have megabytes of logs. So, supposably you develope some application which should know all action...
Published ago by ochirkov

In this article we will parse our apache logs from previous topic by logstash. In a nutshell about logstash config format...    Logstash config consist of three main part:   input {} filter {} output {}   Input field can contain few inputs. It could be stdin, tcp, udp, relp or file. For tests you could use tcp input and stdout output like below:   input { tcp { port => 3333 } } output { stdout { codec => json debug => true } }   Start your logstash with this config and do next on another machine:   nc -v your_logstash_machine_addres...
Published ago by ochirkov

Hi everyone. In this topic I put grok's regexes. Actually you could find them on GitHub but it is comfortably when you read about grok and could look to regexes at this moment. So, list of regexes: USERNAME [a-zA-Z0-9_-]+ USER %{USERNAME} INT (?:[+-]?(?:[0-9]+)) BASE10NUM (?<![0-9.+-])(?>[+-]?(?:(?:[0-9]+(?:\.[0-9]+)?)|(?:\.[0-9]+))) NUMBER (?:%{BASE10NUM}) BASE16NUM (?<![0-9A-Fa-f])(?:[+-]?(?:0x)?(?:[0-9A-Fa-f]+)) BASE16FLOAT \b(?<![0-9A-Fa-f.])(?:[+-]?(?:0x)?(?:(?:[0-9A-Fa-f]+(?:\.[0-9A-Fa-f]*)?)|(?:\.[0-9A-Fa-f]+)))\b POSINT \b(?:[1-9][0-9]*)\b NONNEGINT \b(?:[0-9]+)\b WORD \b\w+\b NOTSPACE \S+ SPACE \s* ...
Published ago by ochirkov

Python
Web development
Review Linux
Administration
Provisioning CVS
Windows AWS
Continuous Integration
NoSql Logging Web Servers FileSystems Scala
accurev ansible apach2 apache api application approaches architecture archivation argparse arguments artifactory automation awk aws backup bash batch-scripting beaver block build built-in caching call captcha cartridge case certificates cgi chain client cloud cloudformation cluster cmd coding collections command-line commands compression conditionals conversion convert cookbook copy counter cron crud css cut cygwin daemon datatype date dd debpackage decorator delattr deploy deque dict distributed-file-systems django DNS domain driver dropbox dump elasticcache elasticsearch encryption exit ext4 extra-tags failover file filename filters for form format freetds functions gerrit getattribute gid git globals glusterfs hardware hook iam indexing inheritance init install job-interview jquery ldap linux list locals logging logs logstash lookup magicmethods mail main-menu metadata metalogger mongodb moosefs mount mssql multiplatform multithreading mysql netcat nginx nosql open-ssh openldap openshift os packaging parse partition path pattern patterns permissions pid pil pip pipe playbook pool post post-commit processes production provisioning proxy putty python python-mysql recursively redirection redis register replication repr restore return review rotation scala script search selenium server setattr settings setup shipper signals singleton slots snapshot socket splunk ssh ssh-key ssl storage str string style subprocess sugar super switch syntactic syntaxhighlighter systeminfo tail tar templatetags time tls tune2fs tuple ubuntu unicode unique unix unixodbc usage usecases uuid uwsgi variable vars version vi virtualenv volume web web-server windows with_items __getattr__