0

I am receiving line by line streaming data from a device with my Python program. Data looks like this:

MSG,4,0,0,80068C,0,2015/01/22,14:40:15.815,2015/01/22,14:40:15.815,,,228.9,212.5,,,-1280,,,,,0
MSG,4,0,0,80068C,0,2015/01/22,14:40:28.722,2015/01/22,14:40:28.722,,,230.5,212.2,,,-1920,,,,,0
MSG,8,0,0,80068C,0,2015/01/22,14:40:34.284,2015/01/22,14:40:34.284,,,,,,,,,,,,0
MSG,8,0,0,80068C,0,2015/01/22,14:40:35.144,2015/01/22,14:40:35.144,,,,,,,,,,,,0
MSG,5,0,0,80068C,0,2015/01/22,14:40:35.144,2015/01/22,14:40:35.144,,5400,,,,,,,,,0,0
MSG,3,0,0,80068C,0,2015/01/22,14:40:36.065,2015/01/22,14:40:36.065,,5375,,,,,,,,,,0
MSG,4,0,0,80068C,0,2015/01/22,14:40:36.644,2015/01/22,14:40:36.644,,,231.6,211.7,,,-1728,,,,,0
MSG,8,0,0,80068C,0,2015/01/22,14:40:37.503,2015/01/22,14:40:37.503,,,,,,,,,,,,0
MSG,4,0,0,80068C,0,2015/01/22,14:40:38.815,2015/01/22,14:40:38.815,,,231.6,211.7,,,-1408,,,,,0

Now I want to connect to SQLite database from the program and save each line to the database in CSV format at the instant they are received.

My Python code is given below:

from socket import *

HOST = 'localhost'
PORT = 30003    #our port from before
ADDR = (HOST,PORT)
BUFSIZE = 4096

sock = socket( AF_INET,SOCK_STREAM)
sock.connect((ADDR))


def readlines(sock, recv_buffer=4096, delim='\n'):
    buffer = ''
    data = True
    while data:
        data = sock.recv(recv_buffer)
        buffer += data

        while buffer.find(delim) != -1:
            line, buffer = buffer.split('\n', 1)
            yield line.strip('\r\n')

    return

for line in readlines(sock):
    print line
Improve this question
4
  • isn't sqlite-database and CSV-format somewhat contradicting? To write CSV you don't need a database. On the other hand to access databases from python take a look at SQLAlchemy, they provide a consistent API on the python side, independent from whatever database system you end up using. Commented Jan 22, 2015 at 15:06
  • Oh sorry!! my post was little confusing!! Actually my data which I recieve is in CSV format. I have created a table in sqlite with a column for each value. Some field have null value. Now each line is considered as a full set of data and to be written to the table ... Thanks Commented Jan 22, 2015 at 15:16
  • If SQLAlchemy is to complex for your need you could also use the build in sqlite3 module. But then you would have to build the SQL statements by hand. Commented Jan 22, 2015 at 15:50
  • possible duplicate of Python and SQLite: insert into table Commented Feb 18, 2015 at 10:07
Related questions
9
Connect to a remote sqlite3 database
0
Python Django sqlite connection
7
Connect to SQLite3 server using PyODBC, Python
Load 7 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.