I wrote a small (~300 line) C program to scrape an email and insert the part I wanted into a Postgres database. I used PCRE for the extraction and ECPG for the Postgres part. ECPG stands for embedded SQL in C for Postgres, and it looks like this:
// Open the storage subsystem.
void open_storage()
{
EXEC SQL BEGIN DECLARE SECTION;
const char* database = DATABASE_NAME;
const char* username = USERNAME;
const char* schema = SCHEMA;
EXEC SQL END DECLARE SECTION;
EXEC SQL WHENEVER SQLERROR CALL quit();
EXEC SQL CONNECT TO :database USER :username;
EXEC SQL SET search_path to :schema;
}
My takeaway was that it was a lot easier than I expected, but it was still kind of a minefield. ECPG isn't widely used or highly intelligent, so there are a lot of stumbling blocks despite the pretty high quality documentation. I would recommend hiding your ECPG behind some functions so you only have to run some of your files through the pre-processor. But it is a neat tool and I could see myself using it again.
No, I signed up for the xe.com daily currency exchange rates email and it occurred to me I could build a database around it. So now I have daily exchange rate data going back a couple years. Thankfully they haven't changed their format. They have a service you can pay for if you want to download this data in bulk but I was being cheap and didn't really have any particular use for the data.
My takeaway was that it was a lot easier than I expected, but it was still kind of a minefield. ECPG isn't widely used or highly intelligent, so there are a lot of stumbling blocks despite the pretty high quality documentation. I would recommend hiding your ECPG behind some functions so you only have to run some of your files through the pre-processor. But it is a neat tool and I could see myself using it again.