Be able to read the postgres data from the inner queries on the C++ code

pgRouting pgrouting-dev
1

Hi every one

I am working on moving the input queries into the C++ code.
The reason for doing this work is:

On the C file:

  • Includes what we call a driver header is linked as C
    • The driver headers are very similar to each other
  • the static process within each C file example are very similar
    • opens the connection with postgres
    • reads the data, (edges sql, restrictions_sql, etc and any array that is on the query)
      • all of these reading is done on the C code which interacts with Postgres, (which is written in C.)
      • Suppose, for simplifying this description, that the data takes x MB in memory
    • calls the pgr_do_function defined on the driver
    • gets the results
    • closes the connection with postgres

On the driver C++ file:

  • code is written in C++
  • Drivers are very similar within each other.
    • Convert C arrays to C++ containers
      • Now memory size is 2x + container overhead
  • Builds the boost graph
    • Now memory size is 3x + container overhead
  • None of this code interacts with Postgres

The ideal situation:

  • Build the boost graph as the data is read.
  • Have C++ templates on the drivers
    • Being so similar that can be done

General steps to reach the ideal situation

  1. Be able to read the postgres data on the C++ code
  2. Create the templates
  3. Build the boost graphs based on the templates needs

Current work

Right now I am working on first step which is, in rough terms, moving the reading of the data to C++, but without moving yet the arrays, only the reading of the SQL queries.

The sketch of the C & C++ driver files for the first step:

On the C file:

  • Includes what we call a driver header linked as C
    • The driver headers are very similar to each other
  • the static process within each C file will still be very similar
    • opens the connection with postgres
    • calls the pgr_do_function defined on the driver
    • gets the results
    • closes the connection with postgres

On the driver C++ file:

  • code is written in C++
  • Drivers will still be very similar within each other.
    • C++ code will interact with postgres
    • Read the data directly into C++ containers
      • Two kinds of data
        • Working on: Data that comes from the inner SQL queries
        • Data that comes from arrays
      • All of these reading will be do with C++ code
      • Now the reading into C++ container takes x MB in memory
  • Builds the boost graph
    • Now memory size is 2x + container overhead

Right now I am working on first step: Be able to read the postgres data from the inner queries on the C++ code

1 Like
2

I am doing also the data that comes from arrays