Results 1 to 7 of 7

Thread: [SOLVED] Migration from PNP to Graphite without data loss

  1. #1

    [SOLVED] Migration from PNP to Graphite without data loss

    Hi,

    I'm wondering if it is possible to migrate from PNP to Graphite without loosing existing graphs ?

    I'm using shinken 1.2.4.

    Thanks

  2. #2
    Shinken project leader
    Join Date
    May 2011
    Location
    Bordeaux (France)
    Posts
    2,131

    Re: Migration from PNP to Graphite without data loss

    You can bulk export raw data from rrd if I'm not wrong, so can be possible to reinsert them into Graphite, but will need some scripting.
    No direct support by personal message. Please open a thread so everyone can see the solution

  3. #3

    Re: Migration from PNP to Graphite without data loss

    Ok i'll try it.

    If I manage to have something i'll post it here

  4. #4

    Re: Migration from PNP to Graphite without data loss

    I have found how to do it !!!

    I'll put the procedure next week !

    8)

  5. #5
    Shinken project leader
    Join Date
    May 2011
    Location
    Bordeaux (France)
    Posts
    2,131

    Re: Migration from PNP to Graphite without data loss

    Great!
    No direct support by personal message. Please open a thread so everyone can see the solution

  6. #6

    Re: Migration from PNP to Graphite without data loss

    Here is how i manage to do it

    It is a bit "naive" way, but i haven't time to make it full automated :


    Pre-requisite :
    • [li]Install Graphite[/li]
      [li]Configure shinken to use graphite[/li]
      [li]DO NOT RESTART SHINKEN NOW[/li]



    Procedure :

    run this command :

    Code:
    echo -e "GET services\n Columns: description host_address perf_data \n OutputFormat: json\n\n" | netcat localhost 50000 > /tmp/services.json
    Copy this script to /tmp/towhisper.sh :

    Code:
    #! /bin/bash
    
    host=$1
    service=$2
    perf=$3
    nb=$4
    
    WSPDIR=/opt/graphite/storage/whisper
    PNPDIR=/usr/local/pnp4nagios/var/perfdata
    
    
    if [[ ! -f $PNPDIR/$host/$service\_$nb.wsp ]]
    then
        rrd2whisper.py $PNPDIR/$host/$service.rrd
    fi
    
    mkdir -p $WSPDIR/$host/$service
    mv $PNPDIR/$host/$service\_$nb.wsp $WSPDIR/$host/$service/$perf.wsp
    chown www-data:www-data $WSPDIR/$host/$service/$perf.wsp
    run this python script from /tmp/ :

    Code:
    import sys
    import json
    sys.path.append("/usr/local/shinken")
    
    from pprint import pprint
    from shinken.misc.perfdata import PerfDatas
    from subprocess import call
    
    
    
    wsp_dir='/opt/graphite/storage/whisper'
    pnp_dir='/usr/local/pnp4nagios/var/perfdata'
    serv_file='/tmp/services.json'
    
    
    
    json_data=open(serv_file)
    data = json.load(json_data)
    for srv in data:
        service=srv[0]
        host=srv[1]
        p = PerfDatas(srv[2])
        i=1
        for perf in p:
            name=perf.name
            name = name.replace("/","_")
            name = name.replace(" ", "_")
            name = name.replace("\\", "_")
            name = name.replace(":", "_")
            print host+" : "+service+" : "+name+" : "+str(i)
            call(["/tmp/towhisper.sh", host, service, name, str(i)])
            i = i + 1

    Run this script from tmp/whisper_host.sh :

    Code:
    WSPDIR=/opt/graphite/storage/whisper
    PNPDIR=/usr/local/pnp4nagios/var/perfdata
    
    find $PNPDIR -name "*\.wsp" -exec rm {} \;
    for h in `/bin/ls $PNPDIR`
    do
    echo $h
    host=$h
    rrd2whisper.py $PNPDIR/$host/_HOST_.rrd >> /dev/null
    mkdir -p $WSPDIR/$host/__HOST__
    mv $PNPDIR/$host/_HOST__1.wsp $WSPDIR/$host/__HOST__/rta.wsp
    mv $PNPDIR/$host/_HOST__2.wsp $WSPDIR/$host/__HOST__/pl.wsp
    mv $PNPDIR/$host/_HOST__3.wsp $WSPDIR/$host/__HOST__/rtmax.wsp
    mv $PNPDIR/$host/_HOST__4.wsp $WSPDIR/$host/__HOST__/rtmin.wsp
    done
    Restart shinken now (arbiter + broker)



    If someone has faith in making automated he's welcome.
    I can help.
    Those scripts are release with no warranty.
    Make sure to read them before using them.
    Feel free to ask any questions.


    And, I know, mixing Python and bash is not good, but i'm better at Bash than python (for now )

  7. #7
    Shinken project leader
    Join Date
    May 2011
    Location
    Bordeaux (France)
    Posts
    2,131

    Re: [SOLVED] Migration from PNP to Graphite without data loss

    Great way

    for the curious: he doesn't restart Shinken so the wisper don't generate void wsp data files, it should starts with the one imported from rrd
    No direct support by personal message. Please open a thread so everyone can see the solution

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •