Drizzle

Merge lp:~hingo/drizzle/drizzle-json_server-keyvalue into lp:drizzle

  1. drizzle-json_server-keyvalue
  2. Merge into 7.2
Proposed by Henrik Ingo
Status: Merged
Merged at revision: 2557
Proposed branch: lp:~hingo/drizzle/drizzle-json_server-keyvalue
Merge into: lp:drizzle
Diff against target: 1576 lines (+1147/-82)
10 files modified
drizzled/plugin/client/cached.h (+7/-4)
drizzled/sql/result_set.h (+41/-11)
drizzled/sql/result_set_meta_data.h (+24/-7)
m4/pandora_have_libevent.m4 (+13/-0)
plugin/json_server/docs/index.rst (+373/-11)
plugin/json_server/json/json_value.cpp (+41/-4)
plugin/json_server/json/value.h (+2/-1)
plugin/json_server/json_server.cc (+637/-43)
plugin/json_server/plugin.ac (+8/-0)
plugin/json_server/plugin.ini (+1/-1)
To merge this branch: bzr merge lp:~hingo/drizzle/drizzle-json_server-keyvalue
Reviewer Review Type Date Requested Status
Henrik Ingo Needs Resubmitting
Review via email: mp+105792@code.launchpad.net

Description of the change

This is 0.2 of json_server plugin. It adds a "pure json" key value api at the uri /json/. (The previous sql-over-http api is at URI /sql/.)

The new functionality still lacks tests, and all of this lacks documentation. We are already working on both, however it would be good to merge this now, because it is easier for us to work against trunk. We will shortly follow up with more tests and docs.

To post a comment you must log in.
Revision history for this message
Brian Aker (brianaker) wrote :
Download full text (97.8 KiB)

Started by upstream project "drizzle-build" build number 1994
Building remotely on ubuntu-11.04-slicehost-174.143.253.46 in workspace /home/jenkins/workspace/drizzle-build-ubuntu-debug

Deleting project workspace... done

Cleaning workspace...
$ bzr branch lp:drizzle/build /home/jenkins/workspace/drizzle-build-ubuntu-debug
You have not informed bzr of your Launchpad ID, and you must do this to
write to Launchpad or access private data. See "bzr help launchpad-login".
Branched 2556 revision(s).
Getting local revision...
$ bzr revision-info -d /home/jenkins/workspace/drizzle-build-ubuntu-debug
info result: bzr revision-info -d /home/jenkins/workspace/drizzle-build-ubuntu-debug returned 0. Command output: "2556 <email address hidden>
" stderr: ""
RevisionState revno:2556 revid:<email address hidden>
[drizzle-build-ubuntu-debug] $ /bin/sh -xe /tmp/hudson1749308941142382677.sh
+ ./config/autorun.sh
: running `python config/pandora-plugin write'
: running `/usr/bin/autoreconf --install --force --verbose -Wall'
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force --warnings=all -I m4 --force
autoreconf: configure.ac: tracing
configure.ac:27: warning: The macro `AC_PROG_LIBTOOL' is obsolete.
configure.ac:27: You should run autoupdate.
aclocal.m4:123: AC_PROG_LIBTOOL is expanded from...
m4/pandora_libtool.m4:6: PANDORA_LIBTOOL is expanded from...
m4/pandora_canonical.m4:41: PANDORA_CANONICAL_TARGET is expanded from...
configure.ac:27: the top level
configure.ac:27: warning: The macro `AC_LANG_SAVE' is obsolete.
configure.ac:27: You should run autoupdate.
../../lib/autoconf/lang.m4:126: AC_LANG_SAVE is expanded from...
../../lib/m4sugar/m4sh.m4:598: AS_IF is expanded from...
../../lib/autoconf/general.m4:2019: AC_CACHE_VAL is expanded from...
../../lib/autoconf/general.m4:2040: AC_CACHE_CHECK is expanded from...
m4/pandora_header_stdcxx_98.m4:21: AC_CXX_HEADER_STDCXX_98 is expanded from...
m4/pandora_check_cxx_standard.m4:6: PANDORA_CHECK_CXX_STANDARD is expanded from...
configure.ac:27: warning: The macro `AC_LANG_CPLUSPLUS' is obsolete.
configure.ac:27: You should run autoupdate.
../../lib/autoconf/c.m4:252: AC_LANG_CPLUSPLUS is expanded from...
configure.ac:27: warning: The macro `AC_TRY_COMPILE' is obsolete.
configure.ac:27: You should run autoupdate.
../../lib/autoconf/general.m4:2602: AC_TRY_COMPILE is expanded from...
configure.ac:27: warning: The macro `AC_LANG_RESTORE' is obsolete.
configure.ac:27: You should run autoupdate.
../../lib/autoconf/lang.m4:135: AC_LANG_RESTORE is expanded from...
m4/pandora_cstdint.m4:12: PANDORA_CXX_CSTDINT is expanded from...
m4/pandora_cinttypes.m4:12: PANDORA_CXX_CINTTYPES is expanded from...
configure.ac:27: warning: The macro `AC_CHECK_LIBM' is obsolete.
configure.ac:27: You should run autoupdate.
aclocal.m4:3301: AC_CHECK_LIBM is expanded from...
m4/pandora_visibility.m4:23: PANDORA_CHECK_VISIBILITY is expanded from...
m4/pandora_visibility.m4:71: PANDORA_ENABLE_VISIBILITY is expanded from...
configure.ac:27: warning: AC_RUN_IFELSE called without default to allow cross compiling
../../lib/autocon...

Revision history for this message
Henrik Ingo (hingo) wrote :

Right. json_server now requires version 2.0 of libevent library. I've now pushed an addition to pandora_have_libevent.m4 that adds a PANDORA_LIBEVENT_RECENT check. Basically, you need Ubuntu Oneiric or newer to have libevent 2.0, otherwise json_server will not be built. (I don't know about Fedora, on Centos I think they still have too old libevent.)

Also pushed the documentation that we finished on Friday.

review: Needs Resubmitting

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'drizzled/plugin/client/cached.h'
2--- drizzled/plugin/client/cached.h 2011-12-23 09:24:47 +0000
3+++ drizzled/plugin/client/cached.h 2012-05-20 15:31:19 +0000
4@@ -47,17 +47,20 @@
5 virtual void sendFields(List<Item>& list)
6 {
7 List<Item>::iterator it(list.begin());
8-
9 column= 0;
10 max_column= 0;
11
12 while (Item* item= it++)
13 {
14- SendField field;
15+ // max_column starts from 0, ColumnCount from 1
16+ _result_set->setColumnCount(max_column+1);
17+ // Get pointer to next column metadata class
18+ SendField field = _result_set->getColumnInfo(max_column);
19 item->make_field(&field);
20+ // Is this necessary or does it just set the same pointer back?
21+ _result_set->setColumnInfo(max_column, field);
22 max_column++;
23 }
24- _result_set->setColumnCount(max_column);
25 // Moved to checkRowBegin()
26 //_result_set->createRow();
27 }
28@@ -75,7 +78,7 @@
29 }
30 }
31
32-virtual void checkRowEnd()
33+ virtual void checkRowEnd()
34 {
35 column++;
36 }
37
38=== modified file 'drizzled/sql/result_set.h'
39--- drizzled/sql/result_set.h 2011-04-21 05:05:53 +0000
40+++ drizzled/sql/result_set.h 2012-05-20 15:31:19 +0000
41@@ -34,6 +34,7 @@
42 #include <drizzled/visibility.h>
43 #include <drizzled/sql/exception.h>
44 #include <drizzled/sql/result_set_meta_data.h>
45+#include <drizzled/field.h>
46 #include <cassert>
47 #include <queue>
48
49@@ -94,19 +95,48 @@
50 bool error() const;
51 sql::Exception getException() const;
52
53- ResultSet(size_t fields) :
54+ ResultSet(size_t columns) :
55 _has_next_been_called(false),
56 _current_row(_results.end()),
57- _meta_data(fields)
58- {
59- }
60-
61- void setColumnCount(size_t fields)
62- {
63- _meta_data.setColumnCount(fields);
64- }
65-
66- ~ResultSet();
67+ _meta_data(columns)
68+ {
69+ }
70+
71+ void setColumnCount(size_t columns)
72+ {
73+ _meta_data.setColumnCount(columns);
74+ }
75+
76+ void setColumnInfo(size_t column_number, const SendField& field)
77+ {
78+ _meta_data.setColumnInfo(column_number, field);
79+ }
80+
81+ /**
82+ * @brief Get object that holds column meta data.
83+ *
84+ * The following info is available:
85+ *
86+ * metadata = rs.getColumnInfo(0);
87+ * metadata.db_name;
88+ * metadata.org_table_name;
89+ * metadata.org_col_name;
90+ * metadata.table_name;
91+ * metadata.col_name;
92+ * metadata.charsetnr;
93+ * metadata.flags;
94+ * metadata.type;
95+ * metadata.length;
96+ * metadata.decimals;
97+ *
98+ * @see drizzled/item.cc to see where these are set.
99+ */
100+ SendField getColumnInfo(size_t column_number)
101+ {
102+ return _meta_data.getColumnInfo(column_number);
103+ }
104+
105+~ResultSet();
106
107 void createRow();
108 void setColumn(size_t column_number, const std::string &arg);
109
110=== modified file 'drizzled/sql/result_set_meta_data.h'
111--- drizzled/sql/result_set_meta_data.h 2011-04-21 05:05:53 +0000
112+++ drizzled/sql/result_set_meta_data.h 2012-05-20 15:31:19 +0000
113@@ -33,10 +33,13 @@
114
115 #include <cassert>
116 #include <queue>
117+#include <vector>
118+#include <drizzled/field.h>
119
120 namespace drizzled {
121 namespace sql {
122
123+
124 class ResultSetMetaData
125 {
126 public:
127@@ -51,17 +54,31 @@
128 ResultSetMetaData(size_t columns) :
129 _columns(columns)
130 {
131- }
132-
133- void setColumnCount(size_t fields)
134- {
135- _columns= fields;
136- }
137-
138+ _columnInfo.resize(columns);
139+ }
140+
141+ void setColumnCount(size_t columns)
142+ {
143+ _columns= columns;
144+ _columnInfo.resize(columns);
145+ }
146+
147+ void setColumnInfo(size_t column_number, const SendField& field)
148+ {
149+ _columnInfo.at(column_number) = field;
150+ }
151+
152+ SendField getColumnInfo(size_t column_number)
153+ {
154+ return _columnInfo.at(column_number);
155+ }
156+
157 private: // Member methods
158
159 private: // Member variables
160 size_t _columns;
161+ typedef std::vector<SendField> ColumnInfoVector;
162+ ColumnInfoVector _columnInfo;
163 };
164
165 std::ostream& operator<<(std::ostream& output, const ResultSetMetaData &result_set);
166
167=== modified file 'm4/pandora_have_libevent.m4'
168--- m4/pandora_have_libevent.m4 2011-03-07 16:42:18 +0000
169+++ m4/pandora_have_libevent.m4 2012-05-20 15:31:19 +0000
170@@ -64,3 +64,16 @@
171 AC_DEFUN([PANDORA_REQUIRE_LIBEVENT],[
172 AC_REQUIRE([_PANDORA_REQUIRE_LIBEVENT])
173 ])
174+
175+AC_DEFUN([PANDORA_LIBEVENT_RECENT],[
176+ dnl FIXME I really wanted to check for existence of EVHTTP_REQ_DELETE,
177+ dnl but autoconf gods were not favorable to me, so the below is all I got
178+ AC_CACHE_CHECK([if libevent is recent enough],
179+ [pandora_cv_libevent_recent],
180+ [AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
181+#include <event2/event.h>
182+#include <event2/http.h>
183+ ]])],
184+ [pandora_cv_libevent_recent=yes],
185+ [pandora_cv_libevent_recent=no])])
186+])
187
188=== modified file 'plugin/json_server/docs/index.rst'
189--- plugin/json_server/docs/index.rst 2011-10-23 05:45:09 +0000
190+++ plugin/json_server/docs/index.rst 2012-05-20 15:31:19 +0000
191@@ -3,7 +3,12 @@
192 JSON Server
193 ===========
194
195-JSON HTTP interface.
196+JSON Server implements a simple HTTP server that allows you to access your
197+Drizzle database with JSON based protocols. Currently two API's are supported:
198+a SQL-over-HTTP protocol allows you to execute any single statement SQL
199+transactions and a pure JSON protocol currently supports storing of JSON
200+documents as blobs in a key-value table.
201+
202
203 .. _json_server_loading:
204
205@@ -51,30 +56,380 @@
206
207 :Scope: Global
208 :Dynamic: No
209- :Option: :option:`--json-server.port`
210
211 Port number to use for connection or 0 for default (port 8086)
212
213-.. _json_server_examples:
214-
215-Examples
216---------
217-
218-Sorry, there are no examples for this plugin.
219+.. _json_server_apis:
220+
221+APIs
222+----
223+
224+JSON Server supports a few APIs that offer different functionalities. Each API
225+is accessed via it's own URI, and parameters can be given in the query string
226+or in the POST data.
227+
228+The APIs are versioned, the version number is prepended to the API name. If
229+functionality is added or changed, it will not be available if an API is
230+accessed via an earlier version number. Finally, the latest version of each API
231+is also available from the root, without any version number.
232+
233+As of this writing, the following APIs exist:
234+
235+.. code-block:: none
236+
237+ /0.1/sql
238+ /0.2/sql
239+ /sql
240+
241+Because the SQL API did not change between 0.1 and 0.2, all of the above URIs
242+are exactly the same.
243+
244+.. code-block:: none
245+
246+ /0.2/json
247+ /json
248+
249+The pure JSON API did not exist in the 0.1 release, as you can see from above.
250+
251+.. code-block:: none
252+
253+ /version
254+ /
255+
256+The ``/version`` URI will return the version of Drizzle (in a JSON document, of
257+course):
258+
259+.. code-block:: none
260+
261+ $ curl http://localhost:8086/version
262+ {
263+ "version" : "7.1.31.2451-snapshot"
264+ }
265+
266+The root URI returns a simple HTML GUI that can be used to test both the SQL and
267+pure JSON APIs. Just point your browser to http://localhost:8086/ and try it!
268+
269+.. _json_server_sql_api:
270+
271+The SQL-over-HTTP API: /sql
272+---------------------------
273+
274+The first API in JSON Server is the SQL-over-HTTP API. It allows you to execute
275+almost any SQL and the result is returned as a 2 dimensional JSON array.
276+
277+On the HTTP level this is a simple API. The method is always ``POST`` and the
278+functionality is determined by the SQL statement you send.
279+
280+.. code-block:: none
281+
282+ POST /sql
283+
284+ SELECT * from test.foo;
285+
286+Returns:
287+
288+.. code-block:: none
289+
290+ {
291+ "query" : "SELECT * from test.foo;\n",
292+ "result_set" : [
293+ [ "1", "Hello Drizzle Day Audience!" ],
294+ [ "2", "this text came in over http" ]
295+ ],
296+ "sqlstate" : "00000"
297+ }
298+
299+The above corresponds to the following query from a drizzle command line:
300+
301+.. code-block:: mysql
302+
303+ drizzle> select * from test.foo;
304+
305++----+-----------------------------+
306+| id | bar |
307++====+=============================+
308+| 1 | Hello Drizzle Day Audience! |
309++----+-----------------------------+
310+| 2 | this text came in over http |
311++----+-----------------------------+
312+
313+
314+.. _json_server_json_api:
315+
316+Pure JSON key-value API: /json
317+------------------------------
318+
319+The pure JSON key-value API is found at the URI ``/json``. It takes a rather
320+opposite approach than the ``/sql`` API. Queries are expressed as JSON query
321+documents, similar to what is found in Metabase, CouchDB or MongoDB. It is not
322+possible to use any SQL.
323+
324+The purpose of the ``/json`` API is to use Drizzle as a key-value document
325+storage. This means that the table layout is determined by the JSON Server
326+module. Therefore, it is not possible for the user to access arbitrary
327+relational tables via the ``/json`` API, rather tables must adhere to the
328+format explained further below, and it must contain valid JSON documents in the
329+data columns.
330+
331+If you post (insert) a document to a table that doesn't exist, it will be
332+automatically created. For this reason, a user mostly doesn't need to even
333+know the specific format of a JSON server table.
334+
335+.. _json_server_json_parameters:
336+
337+Parameters
338+^^^^^^^^^^
339+
340+Following parameters can be passed in the URI query string:
341+
342+.. _json_server_json_parameters_id:
343+
344+``_id``
345+
346+ :Type: Unsigned integer
347+ :Mandatory: No
348+ :Default:
349+
350+ Optionally, a user may also specify the _id value which is requested.
351+ Typically this is given in the JSON query document instead. If both are given
352+ the _id value in the query document has precedence.
353+
354+.. _json_server_json_parameters_query:
355+
356+``query``
357+
358+ :Type: JSON query document
359+ :Mandatory: No
360+ :Default:
361+
362+ A JSON document, the so called *query document*. This document specifies
363+ which records/documents to return from the database. Currently it is only
364+ possible to query for a single value by the primary key, which is
365+ called ``_id``. Any other fields in the query document will be ignored.
366+
367+ The query parameter is used for GET, PUT and DELETE where it is passed in
368+ URL encoded form in the URI query string. For POST requests the query
369+ document is passed as the POST data. (In that case only the query document
370+ is passed, there is no ``query=`` part, in other words the data is never
371+ encoded in www-form-urlencoded format.)
372+
373+ Example query document:
374+
375+ .. code-block:: none
376+
377+ { "_id" : 1 }
378+
379+.. _json_server_json_parameters_schema:
380+
381+``schema``
382+
383+ :Type: String
384+ :Mandatory: No
385+ :Default: test
386+
387+ Name of the schema which we are querying. The schema must exist.
388+
389+.. _json_server_parameters_table:
390+
391+``table``
392+
393+ :Type: String
394+ :Mandatory: No
395+ :Default: jsonkv
396+
397+ Name of the table which we are querying. For POST requests, if the table
398+ doesn't exist, it will be automatically created. For other requests the
399+ table must exist, otherwise an error is returned.
400+
401+POSTing a document
402+^^^^^^^^^^^^^^^^^^
403+
404+.. code-block:: none
405+
406+ POST /json?schema=test&table=people HTTP/1.1
407+
408+ {
409+ "_id" : 2,
410+ "document" : { "firstname" : "Henrik", "lastname" : "Ingo", "age" : 35}
411+ }
412+
413+Returns:
414+
415+.. code-block:: none
416+
417+ HTTP/1.1 200 OK
418+ Content-Type: text/html
419+
420+ {
421+ "query" : {
422+ "_id" : 2,
423+ "document" : {
424+ "age" : 35,
425+ "firstname" : "Henrik",
426+ "lastname" : "Ingo"
427+ }
428+ },
429+ "sqlstate" : "00000"
430+ }
431+
432+
433+(The use of Content-type: text/html is considered a bug and will be
434+fixed in a future version.)
435+
436+Under the hood, this has inserted the following record into a table "jsontable":
437+
438+.. code-block:: mysql
439+
440+ drizzle> select * from people where _id=2;
441+
442++-----+--------------------------+
443+| _id | document |
444++=====+==========================+
445+| 2 |{ |
446+| |"age" : 35, |
447+| |"firstname" : "Henrik", |
448+| |"lastname" : "Ingo" |
449+| |} |
450++-----+--------------------------+
451+
452+The ``_id`` field is always present. If it isn't specified, an auto_increment
453+value will be generated. If a record with the given ``_id`` already exists in
454+the table, the record will be updated (using REPLACE INTO).
455+
456+In addition there are one or more columns of type TEXT.
457+The column name(s) corresponds to the top level key(s) that were specified in the
458+POSTed JSON document. You can use any name(s) for the top level key(s), but
459+the name ``document`` is commonly used as a generic name. The contents of such a
460+column is the value of the corresponding top level key and has to be valid JSON.
461+
462+A table of this format is automatically created when the first document is
463+POSTed to the table. This means that the column names are defined from the top
464+level key(s) of that first document and future JSON documents must use the same
465+top level key(s). Below the top level key(s) the JSON document can be of any
466+arbitrary structure. A common practice is to always use ``_id`` and ``document``
467+as the top level keys, and place the actual JSON document, which can be of
468+arbitrary structure, under the ``document`` key.
469+
470+
471+GET a document
472+^^^^^^^^^^^^^^
473+
474+The equivalent of an SQL SELECT is HTTP GET.
475+
476+Below we use the query document ``{"_id" : 1 }`` in URL encoded form:
477+
478+.. code-block:: none
479+
480+ GET /json?schema=test&table=people&query=%7B%22_id%22%20%3A%201%7D%0A
481+
482+Returns
483+
484+.. code-block:: none
485+
486+ HTTP/1.0 200 OK
487+ Content-Type: text/html
488+
489+ {
490+ "query" : {
491+ "_id" : 1
492+ },
493+ "result_set" : [
494+ {
495+ "_id" : 1,
496+ "document" : {
497+ "age" : 21,
498+ "firstname" : "Mohit",
499+ "lastname" : "Srivastava"
500+ }
501+ }
502+ ],
503+ "sqlstate" : "00000"
504+ }
505+
506+It is also allowed to specify the ``_id`` as a URI query string parameter and
507+omit the query document:
508+
509+.. code-block:: none
510+
511+ GET /json?schema=test&table=people&_id=1
512+
513+If both are specified, the query document takes precedence.
514+
515+Finally, it is possible to issue a GET request to a table without specifying
516+neither the ``_id`` parameter or a query document. In this case all records of
517+the whole table is returned.
518+
519+
520+Updating a record
521+^^^^^^^^^^^^^^^^^
522+
523+To update a record, POST new version of json document with same ``_id`` as an
524+already existing record.
525+
526+(PUT is currently not supported, instead POST is used for both inserting and
527+updating.)
528+
529+Deleting a record
530+^^^^^^^^^^^^^^^^^
531+
532+Below we use the query document ``{"_id" : 1 }`` in URL encoded form:
533+
534+.. code-block:: none
535+
536+ DELETE http://14.139.228.217:8086/json?schema=test&table=people&query=%7B%22_id%22%20%3A%201%7D
537+
538+Returns:
539+
540+.. code-block:: none
541+
542+ HTTP/1.0 200 OK
543+ Content-Type: text/html
544+
545+ {
546+ "query" : {
547+ "_id" : 1
548+ },
549+ "sqlstate" : "00000"
550+ }
551+
552+It is also allowed to specify the ``_id`` as a URI query string parameter and
553+omit the query document:
554+
555+.. code-block:: none
556+
557+ DELETE /json?schema=test&table=people&_id=1
558+
559+If both are specified, the query document takes precedence.
560+
561+.. _json_server_limitations:
562+
563+Limitations
564+^^^^^^^^^^^
565+
566+The ``/sql`` and ``/json`` APIs are both feature complete, yet JSON Server is
567+still an experimental module. There are known crashes, the module is still
568+single threaded and there is no authentication... and that's just a start!
569+These limitations are being worked on. For a full list of the current state of
570+JSON Server, please follow
571+`this launchpad blueprint <https://blueprints.launchpad.net/drizzle/+spec/json-server>`_.
572+
573+An inherent limitation is that each HTTP request is its own transaction. While
574+it would be possible to support maintaining a complex SQL transaction over the
575+span of multiple HTTP requests, we currently do not plan to support that.
576
577 .. _json_server_authors:
578
579 Authors
580 -------
581
582-Stewart Smith
583+Stewart Smith, Henrik Ingo, Mohit Srivastava
584
585 .. _json_server_version:
586
587 Version
588 -------
589
590-This documentation applies to **json_server 0.1**.
591+This documentation applies to **json_server 0.2**.
592
593 To see which version of the plugin a Drizzle server is running, execute:
594
595@@ -87,4 +442,11 @@
596
597 v0.1
598 ^^^^
599-* First release.
600+* /sql API
601+* Simple web based GUI at /
602+* /version API
603+
604+v0.2
605+^^^^
606+* /json API supporting pure JSON key-value operations (POST, GET, DELETE)
607+* Automatic creation of table on first post.
608
609=== modified file 'plugin/json_server/json/json_value.cpp'
610--- plugin/json_server/json/json_value.cpp 2011-08-20 13:41:35 +0000
611+++ plugin/json_server/json/json_value.cpp 2012-05-20 15:31:19 +0000
612@@ -40,6 +40,7 @@
613 #include <plugin/json_server/json/value.h>
614 #include <plugin/json_server/json/writer.h>
615
616+#include <cstdio>
617 #include <cassert>
618 #include <cstring>
619 #include <iostream>
620@@ -308,6 +309,7 @@
621 # ifdef JSON_VALUE_USE_INTERNAL_MAP
622 , itemIsUsed_( 0 )
623 #endif
624+ , value_as_string_( 0 )
625 {
626 switch ( type_arg )
627 {
628@@ -351,6 +353,7 @@
629 # ifdef JSON_VALUE_USE_INTERNAL_MAP
630 , itemIsUsed_( 0 )
631 #endif
632+ , value_as_string_( 0 )
633 {
634 value_.int_ = value;
635 }
636@@ -362,6 +365,7 @@
637 # ifdef JSON_VALUE_USE_INTERNAL_MAP
638 , itemIsUsed_( 0 )
639 #endif
640+ , value_as_string_( 0 )
641 {
642 value_.uint_ = value;
643 }
644@@ -372,6 +376,7 @@
645 # ifdef JSON_VALUE_USE_INTERNAL_MAP
646 , itemIsUsed_( 0 )
647 #endif
648+ , value_as_string_( 0 )
649 {
650 value_.real_ = value;
651 }
652@@ -383,6 +388,7 @@
653 # ifdef JSON_VALUE_USE_INTERNAL_MAP
654 , itemIsUsed_( 0 )
655 #endif
656+ , value_as_string_( 0 )
657 {
658 value_.string_ = valueAllocator()->duplicateStringValue( value );
659 }
660@@ -396,6 +402,7 @@
661 # ifdef JSON_VALUE_USE_INTERNAL_MAP
662 , itemIsUsed_( 0 )
663 #endif
664+ , value_as_string_( 0 )
665 {
666 value_.string_ = valueAllocator()->duplicateStringValue( beginValue,
667 UInt(endValue - beginValue) );
668@@ -409,6 +416,7 @@
669 # ifdef JSON_VALUE_USE_INTERNAL_MAP
670 , itemIsUsed_( 0 )
671 #endif
672+ , value_as_string_( 0 )
673 {
674 value_.string_ = valueAllocator()->duplicateStringValue( value.c_str(),
675 (unsigned int)value.length() );
676@@ -422,6 +430,7 @@
677 # ifdef JSON_VALUE_USE_INTERNAL_MAP
678 , itemIsUsed_( 0 )
679 #endif
680+ , value_as_string_( 0 )
681 {
682 value_.string_ = const_cast<char *>( value.c_str() );
683 }
684@@ -435,6 +444,7 @@
685 # ifdef JSON_VALUE_USE_INTERNAL_MAP
686 , itemIsUsed_( 0 )
687 #endif
688+ , value_as_string_( 0 )
689 {
690 value_.string_ = valueAllocator()->duplicateStringValue( value, value.length() );
691 }
692@@ -446,6 +456,7 @@
693 # ifdef JSON_VALUE_USE_INTERNAL_MAP
694 , itemIsUsed_( 0 )
695 #endif
696+ , value_as_string_( 0 )
697 {
698 value_.bool_ = value;
699 }
700@@ -457,6 +468,7 @@
701 # ifdef JSON_VALUE_USE_INTERNAL_MAP
702 , itemIsUsed_( 0 )
703 #endif
704+ , value_as_string_( 0 )
705 {
706 switch ( type_ )
707 {
708@@ -504,7 +516,6 @@
709 }
710 }
711
712-
713 Value::~Value()
714 {
715 switch ( type_ )
716@@ -514,7 +525,9 @@
717 case uintValue:
718 case realValue:
719 case booleanValue:
720- break;
721+ if ( value_as_string_ )
722+ valueAllocator()->releaseStringValue( value_as_string_ );
723+ break;
724 case stringValue:
725 if ( allocated_ )
726 valueAllocator()->releaseStringValue( value_.string_ );
727@@ -715,9 +728,12 @@
728 return value_.string_;
729 }
730
731-
732+/**
733+ * If type_ is not stringValue, we will here convert other values to value_as_string_ field,
734+ * then return it.
735+ */
736 std::string
737-Value::asString() const
738+Value::asString()
739 {
740 switch ( type_ )
741 {
742@@ -728,8 +744,29 @@
743 case booleanValue:
744 return value_.bool_ ? "true" : "false";
745 case intValue:
746+ if(!value_as_string_)
747+ {
748+ char buf[64];
749+ sprintf(buf, "%d", value_.int_);
750+ value_as_string_ = valueAllocator()->duplicateStringValue( buf );
751+ }
752+ return value_as_string_;
753 case uintValue:
754+ if(!value_as_string_)
755+ {
756+ char buf[64];
757+ sprintf(buf, "%d", value_.uint_);
758+ value_as_string_ = valueAllocator()->duplicateStringValue( buf );
759+ }
760+ return value_as_string_;
761 case realValue:
762+ if(!value_as_string_)
763+ {
764+ char buf[256];
765+ sprintf(buf, "%f", value_.real_);
766+ value_as_string_ = valueAllocator()->duplicateStringValue( buf );
767+ }
768+ return value_as_string_;
769 case arrayValue:
770 case objectValue:
771 JSON_ASSERT_MESSAGE( false, "Type is not convertible to string" );
772
773=== modified file 'plugin/json_server/json/value.h'
774--- plugin/json_server/json/value.h 2011-08-19 09:52:50 +0000
775+++ plugin/json_server/json/value.h 2012-05-20 15:31:19 +0000
776@@ -263,7 +263,7 @@
777 int compare( const Value &other );
778
779 const char *asCString() const;
780- std::string asString() const;
781+ std::string asString();
782 # ifdef JSON_USE_CPPTL
783 CppTL::ConstString asConstString() const;
784 # endif
785@@ -481,6 +481,7 @@
786 int memberNameIsStatic_ : 1; // used by the ValueInternalMap container.
787 # endif
788 CommentInfo *comments_;
789+ char *value_as_string_; // Used when asString is called on non-string types.
790 };
791
792
793
794=== modified file 'plugin/json_server/json_server.cc'
795--- plugin/json_server/json_server.cc 2012-01-16 02:37:54 +0000
796+++ plugin/json_server/json_server.cc 2012-05-20 15:31:19 +0000
797@@ -1,7 +1,7 @@
798 /* - mode: c; c-basic-offset: 2; indent-tabs-mode: nil; -*-
799 * vim:expandtab:shiftwidth=2:tabstop=2:smarttab:
800 *
801- * Copyright (C) 2011 Stewart Smith
802+ * Copyright (C) 2011 Stewart Smith, Henrik Ingo, Mohit Srivastava
803 *
804 * This program is free software; you can redistribute it and/or modify
805 * it under the terms of the GNU General Public License as published by
806@@ -17,7 +17,21 @@
807 * along with this program; if not, write to the Free Software
808 * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
809 */
810-
811+/**
812+ * @file Implements an HTTP server that will parse JSON and SQL queries
813+ *
814+ * @todo Refactoring ideas:
815+ * - Anything HTML should really be a separate file, not strings embedded
816+ * in C++.
817+ * - Put all json handling into try/catch blocks, the parser likes to throw
818+ * exceptions which crash drizzled if not caught.
819+ * - Need to implement a worker thread pool. Make workers proper OO classes.
820+ *
821+ * @todo Implement HTTP response codes other than just 200 as defined in
822+ * http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html
823+ *
824+ * @todo Shouldn't we be using event2/http.h? Why does this even work without it?
825+ */
826
827 #include <config.h>
828
829@@ -76,7 +90,11 @@
830 extern "C" void process_root_request(struct evhttp_request *req, void* );
831 extern "C" void process_api01_version_req(struct evhttp_request *req, void* );
832 extern "C" void process_api01_sql_req(struct evhttp_request *req, void* );
833-
834+extern "C" void process_api02_json_req(struct evhttp_request *req, void* );
835+extern "C" void process_api02_json_get_req(struct evhttp_request *req, void* );
836+extern "C" void process_api02_json_post_req(struct evhttp_request *req, void* );
837+/* extern "C" void process_api02_json_put_req(struct evhttp_request *req, void* ); */
838+extern "C" void process_api02_json_delete_req(struct evhttp_request *req, void* );
839 extern "C" void process_request(struct evhttp_request *req, void* )
840 {
841 struct evbuffer *buf = evbuffer_new();
842@@ -92,40 +110,78 @@
843
844 std::string output;
845
846- output.append("<html><head><title>JSON DATABASE interface demo</title></head>"
847- "<body>"
848- "<script lang=\"javascript\">"
849- "function to_table(obj) {"
850- " var str = '<table>';"
851- "for (var r=0; r< obj.length; r++) {"
852- " str+='<tr>';"
853- " for (var c=0; c < obj[r].length; c++) {"
854- " str+= '<td>' + obj[r][c] + '</td>';"
855- " }"
856- " str+='</tr>';"
857- "}"
858- "str+='</table>';"
859- "return str;"
860- "}"
861- "function run_query()\n"
862- "{"
863- "var url = document.getElementById(\"baseurl\").innerHTML;\n"
864- "var query= document.getElementById(\"query\").value;\n"
865+ output.append("<html><head><title>JSON DATABASE interface demo</title></head>\n"
866+ "<body>\n"
867+ "<script lang=\"javascript\">\n"
868+ "function to_table(obj) {\n"
869+ " var str = '<table border=\"1\">';\n"
870+ "for (var r=0; r< obj.length; r++) {\n"
871+ " str+='<tr>';\n"
872+ " for (var c=0; c < obj[r].length; c++) {\n"
873+ " str+= '<td>' + obj[r][c] + '</td>';\n"
874+ " }\n"
875+ " str+='</tr>';\n"
876+ "}\n"
877+ "str+='</table>';\n"
878+ "return str;\n"
879+ "}\n"
880+ "function to_table_from_json(obj) {\n"
881+ " var str = '<table border=\"1\">';\n"
882+ "for (var r=0; r< obj.length; r++) {\n"
883+ " str+='<tr>';\n"
884+ " str+='<td>' + obj[r]['_id'] + '</td>';\n"
885+ " str+='<td>' + JSON.stringify(obj[r]['document']) + '</td>';\n"
886+ " str+='</tr>';\n"
887+ "}\n"
888+ "str+='</table>';\n"
889+ "return str;\n"
890+ "}\n"
891+ "function run_sql_query()\n"
892+ "{\n"
893+ "var url = window.location;\n"
894+ "var query= document.getElementById(\"sql_query\").value;\n"
895 "var xmlHttp = new XMLHttpRequest();\n"
896 "xmlHttp.onreadystatechange = function () {\n"
897+ "document.getElementById(\"responseText\").value = xmlHttp.responseText;\n"
898 "if (xmlHttp.readyState == 4 && xmlHttp.status == 200) {\n"
899 "var info = eval ( \"(\" + xmlHttp.responseText + \")\" );\n"
900 "document.getElementById( \"resultset\").innerHTML= to_table(info.result_set);\n"
901 "}\n"
902 "};\n"
903- "xmlHttp.open(\"POST\", url + \"/0.1/sql\", true);"
904- "xmlHttp.send(query);"
905- "}"
906+ "xmlHttp.open(\"POST\", url + \"sql\", true);\n"
907+ "xmlHttp.send(query);\n"
908+ "}\n"
909+ "\n\n"
910+ "function run_json_query()\n"
911+ "{\n"
912+//"alert('run_json_query');"
913+ "var url = window.location;\n"
914+ "var method= document.getElementById(\"json_method\").value;\n"
915+ "var query= document.getElementById(\"json_query\").value;\n"
916+ "var schema= document.getElementById(\"schema\").value;\n"
917+ "var table= document.getElementById(\"table\").value;\n"
918+ "var xmlHttp = new XMLHttpRequest();\n"
919+ "xmlHttp.onreadystatechange = function () {\n"
920+//"alert(xmlHttp.responseText);"
921+ "document.getElementById(\"responseText\").value = xmlHttp.responseText;\n"
922+ "if (xmlHttp.readyState == 4 && xmlHttp.status == 200) {\n"
923+ "var info = eval ( \"(\" + xmlHttp.responseText + \")\" );\n"
924+ "document.getElementById( \"resultset\").innerHTML= to_table_from_json(info.result_set);\n"
925+ "}\n"
926+ "};\n"
927+ "if( method == \"POST\" ) {\n"
928+ "xmlHttp.open(method, url + \"json?schema=\" + schema + \"&table=\" + table, true);\n"
929+ "xmlHttp.send(query);\n"
930+ "} else {\n"
931+ "xmlHttp.open(method, url + \"json?schema=\" + schema + \"&table=\" + table + \"&query=\" + encodeURIComponent(query), true);\n"
932+ "xmlHttp.send();\n"
933+ "}\n"
934+ "}\n"
935 "\n\n"
936 "function update_version()\n"
937- "{drizzle_version(document.getElementById(\"baseurl\").innerHTML);}\n\n"
938- "function drizzle_version($url)"
939- "{"
940+ "{drizzle_version(window.location);}\n\n"
941+ "function drizzle_version($url)\n"
942+ "{\n"
943 "var xmlHttp = new XMLHttpRequest();\n"
944 "xmlHttp.onreadystatechange = function () {\n"
945 "if (xmlHttp.readyState == 4 && xmlHttp.status == 200) {\n"
946@@ -133,19 +189,32 @@
947 "document.getElementById( \"drizzleversion\").innerHTML= info.version;\n"
948 "}\n"
949 "};\n"
950- "xmlHttp.open(\"GET\", $url + \"/0.1/version\", true);"
951- "xmlHttp.send(null);"
952- "}"
953- "</script>"
954- "<p>Drizzle Server at: <a id=\"baseurl\">http://localhost:8765</a></p>"
955- "<p>Drizzle server version: <a id=\"drizzleversion\"></a></p>"
956- "<p><textarea rows=\"3\" cols=\"40\" id=\"query\">"
957- "SELECT * from DATA_DICTIONARY.GLOBAL_STATUS;"
958- "</textarea>"
959- "<button type=\"button\" onclick=\"run_query();\">Execute Query</button>"
960- "<div id=\"resultset\"/>"
961- "<script lang=\"javascript\">update_version(); run_query();</script>"
962- "</body></html>");
963+ "xmlHttp.open(\"GET\", $url + \"version\", true);\n"
964+ "xmlHttp.send(null);\n"
965+ "}\n"
966+ "</script>\n"
967+ "<p>Drizzle server version: <a id=\"drizzleversion\"></a></p>\n"
968+ "<p><textarea rows=\"3\" cols=\"80\" id=\"sql_query\">\n"
969+ "SELECT * from DATA_DICTIONARY.GLOBAL_STATUS;\n"
970+ "</textarea>\n"
971+ "<button type=\"button\" onclick=\"run_sql_query();\">Execute SQL Query</button>\n"
972+ "</p><p>\n"
973+ "<textarea rows=\"8\" cols=\"80\" id=\"json_query\">\n"
974+ "{\"_id\" : 1}\n"
975+ "</textarea>\n"
976+ "<button type=\"button\" onclick=\"run_json_query();\">Execute JSON Query</button>\n"
977+ "<br />\n"
978+ "<select id=\"json_method\"><option value=\"GET\">GET</option>"
979+ "<option value=\"POST\">POST</option>"
980+ "<option value=\"PUT\">PUT</option>"
981+ "<option value=\"DELETE\">DELETE</option></select>"
982+ "<script lang=\"javascript\">document.write(window.location);</script>json?schema=\n"
983+ "<input type=\"text\" id=\"schema\" value=\"test\"/>"
984+ "&amp;table=<input type=\"text\" id=\"table\" value=\"jsonkv\"/>\n"
985+ "</p><hr />\n<div id=\"resultset\"></div>\n"
986+ "<hr /><p><textarea rows=\"12\" cols=\"80\" id=\"responseText\" ></textarea></p>"
987+ "<script lang=\"javascript\">update_version(); run_sql_query();</script>\n"
988+ "</body></html>\n");
989
990 evbuffer_add(buf, output.c_str(), output.length());
991 evhttp_send_reply(req, HTTP_OK, "OK", buf);
992@@ -203,6 +272,7 @@
993 {
994 root["error_message"]= exception.getErrorMessage();
995 root["error_code"]= exception.getErrorCode();
996+ root["schema"]= "test";
997 }
998
999 while (result_set.next())
1000@@ -227,6 +297,515 @@
1001 evhttp_send_reply(req, HTTP_OK, "OK", buf);
1002 }
1003
1004+extern "C" void process_api02_json_req(struct evhttp_request *req, void* )
1005+{
1006+ if( req->type == EVHTTP_REQ_GET )
1007+ {
1008+ process_api02_json_get_req( req, NULL);
1009+// } elseif ( req->type == EVHTTP_REQ_PUT ) {
1010+ //process_api02_json_put_req( req, NULL);
1011+ } else if ( req->type == EVHTTP_REQ_POST ) {
1012+ process_api02_json_post_req( req, NULL);
1013+ } else if ( req->type == EVHTTP_REQ_DELETE ) {
1014+ process_api02_json_delete_req( req, NULL);
1015+ }
1016+}
1017+
1018+/**
1019+ * Transform a HTTP GET to SELECT and return results based on input json document
1020+ *
1021+ * @todo allow DBA to set default schema (also in post,del methods)
1022+ * @todo allow DBA to set whether to use strict mode for parsing json (should get rid of white space), especially for POST of course.
1023+ *
1024+ * @param req should contain a "table" parameter in request uri. "query", "_id" and "schema" are optional.
1025+ * @return a json document is returned to client with evhttp_send_reply()
1026+ */
1027+void process_api02_json_get_req(struct evhttp_request *req, void* )
1028+{
1029+ int http_response_code = HTTP_OK;
1030+ const char *http_response_text;
1031+ http_response_text = "OK";
1032+
1033+ struct evbuffer *buf = evbuffer_new();
1034+ if (buf == NULL) return;
1035+
1036+ Json::Value json_out;
1037+
1038+ std::string input;
1039+ // Schema and table are given in request uri.
1040+ // TODO: If we want to be really NoSQL, we will some day allow to use synonyms like "collection" instead of "table".
1041+ // For GET, also the query is in the uri
1042+ const char *schema;
1043+ const char *table;
1044+ const char *query;
1045+ const char *id;
1046+ evhttp_parse_query(evhttp_request_uri(req), req->input_headers);
1047+ schema = (char *)evhttp_find_header(req->input_headers, "schema");
1048+ table = (char *)evhttp_find_header(req->input_headers, "table");
1049+ query = (char *)evhttp_find_header(req->input_headers, "query");
1050+ id = (char *)evhttp_find_header(req->input_headers, "_id");
1051+
1052+ // query can be null if _id was given
1053+ if ( query == NULL || strcmp(query, "") == 0 )
1054+ {
1055+ // Empty JSON object
1056+ query = "{}";
1057+ }
1058+ input.append(query, strlen(query));
1059+
1060+ // Set test as default schema
1061+ if ( !strcmp( schema, "") || schema == NULL)
1062+ {
1063+ schema = "test";
1064+ }
1065+
1066+ // Parse "input" into "json_in".
1067+ Json::Value json_in;
1068+ Json::Features json_conf;
1069+ Json::Reader reader(json_conf);
1070+ bool retval = reader.parse(input, json_in);
1071+ if (retval != true) {
1072+ json_out["error_type"]="json error";
1073+ json_out["error_message"]= reader.getFormatedErrorMessages();
1074+ }
1075+ else if (strcmp( table, "") == 0 || table == NULL) {
1076+ json_out["error_type"]="http error";
1077+ json_out["error_message"]= "You must specify \"table\" in the request uri query string.";
1078+ http_response_code = HTTP_NOTFOUND;
1079+ http_response_text = "You must specify \"table\" in the request uri query string.";
1080+ }
1081+ else {
1082+ // It is allowed to specify _id in the uri and leave it out from the json query.
1083+ // In that case we put the value from uri into json_in here.
1084+ // If both are specified, the one existing in json_in wins. (This is still valid, no error.)
1085+ if ( ! json_in["_id"].asBool() )
1086+ {
1087+ if( id ) {
1088+ json_in["_id"] = (Json::Value::UInt) atol(id);
1089+ }
1090+ }
1091+
1092+ // TODO: In a later stage we'll allow the situation where _id isn't given but some other column for where.
1093+ // TODO: Need to do json_in[].type() first and juggle it from there to be safe. See json/value.h
1094+ // TODO: Don't SELECT * but only fields given in json query document
1095+ char sqlformat[1024];;
1096+ char buffer[1024];
1097+ if ( json_in["_id"].asBool() )
1098+ {
1099+ // Now we build an SQL query, using _id from json_in
1100+ sprintf(sqlformat, "%s", "SELECT * FROM `%s`.`%s` WHERE _id=%i;");
1101+ sprintf(buffer, sqlformat, schema, table, json_in["_id"].asInt());
1102+ }
1103+ else {
1104+ // If neither _id nor query are given, we return the full table. (No error, maybe this is what you really want? Blame yourself.)
1105+ sprintf(sqlformat, "%s", "SELECT * FROM `%s`.`%s`;");
1106+ sprintf(buffer, sqlformat, schema, table);
1107+ }
1108+
1109+ std::string sql = "";
1110+ sql.append(buffer, strlen(buffer));
1111+
1112+ // We have sql string. Use Execute API to run it and convert results back to JSON.
1113+ drizzled::Session::shared_ptr _session= drizzled::Session::make_shared(drizzled::plugin::Listen::getNullClient(),
1114+ drizzled::catalog::local());
1115+ drizzled::identifier::user::mptr user_id= identifier::User::make_shared();
1116+ user_id->setUser("");
1117+ _session->setUser(user_id);
1118+ //_session->set_schema("test");
1119+
1120+ drizzled::Execute execute(*(_session.get()), true);
1121+
1122+ drizzled::sql::ResultSet result_set(1);
1123+
1124+ /* Execute wraps the SQL to run within a transaction */
1125+ execute.run(sql, result_set);
1126+ drizzled::sql::Exception exception= result_set.getException();
1127+
1128+ drizzled::error_t err= exception.getErrorCode();
1129+
1130+ json_out["sqlstate"]= exception.getSQLState();
1131+
1132+ if ((err != drizzled::EE_OK) && (err != drizzled::ER_EMPTY_QUERY))
1133+ {
1134+ json_out["error_type"]="sql error";
1135+ json_out["error_message"]= exception.getErrorMessage();
1136+ json_out["error_code"]= exception.getErrorCode();
1137+ json_out["internal_sql_query"]= sql;
1138+ json_out["schema"]= "test";
1139+ }
1140+
1141+ while (result_set.next())
1142+ {
1143+ Json::Value json_row;
1144+ bool got_error = false;
1145+ for (size_t x= 0; x < result_set.getMetaData().getColumnCount() && got_error == false; x++)
1146+ {
1147+ if (not result_set.isNull(x))
1148+ {
1149+ // The values are now serialized json. We must first
1150+ // parse them to make them part of this structure, only to immediately
1151+ // serialize them again in the next step. For large json documents
1152+ // stored into the blob this must be very, very inefficient.
1153+ // TODO: Implement a smarter way to push the blob value directly to the client. Probably need to hand code some string appending magic.
1154+ // TODO: Massimo knows of a library to create JSON in streaming mode.
1155+ Json::Value json_doc;
1156+ Json::Reader readrow(json_conf);
1157+ std::string col_name = result_set.getColumnInfo(x).col_name;
1158+ bool r = readrow.parse(result_set.getString(x), json_doc);
1159+ if (r != true) {
1160+ json_out["error_type"]="json parse error on row value";
1161+ json_out["error_internal_sql_column"]=col_name;
1162+ json_out["error_message"]= reader.getFormatedErrorMessages();
1163+ // Just put the string there as it is, better than nothing.
1164+ json_row[col_name]= result_set.getString(x);
1165+ got_error=true;
1166+ break;
1167+ }
1168+ else {
1169+ json_row[col_name]= json_doc;
1170+ }
1171+ }
1172+ }
1173+ // When done, append this to result set tree
1174+ json_out["result_set"].append(json_row);
1175+ }
1176+
1177+ json_out["query"]= json_in;
1178+ }
1179+ // Return either the results or an error message, in json.
1180+ Json::StyledWriter writer;
1181+ std::string output= writer.write(json_out);
1182+ evbuffer_add(buf, output.c_str(), output.length());
1183+ evhttp_send_reply(req, http_response_code, http_response_text, buf);
1184+}
1185+
1186+/**
1187+ * Input json document or update existing one from HTTP POST (and PUT?).
1188+ *
1189+ * If json document specifies _id field, then record is updated. If it doesn't
1190+ * exist, then a new record is created with that _id.
1191+ *
1192+ * If _id field is not specified, then a new record is created using
1193+ * auto_increment value. The _id of the created value is returned in the http
1194+ * response.
1195+ *
1196+ * @todo If there are multiple errors, last one overwrites the previous in json_out. Make them lists.
1197+ *
1198+ * @param req should contain a "table" parameter in request uri. "schema" is optional.
1199+ * @return a json document is returned to client with evhttp_send_reply()
1200+ */
1201+void process_api02_json_post_req(struct evhttp_request *req, void* )
1202+{
1203+ bool table_exists = true;
1204+ Json::Value json_out;
1205+
1206+ struct evbuffer *buf = evbuffer_new();
1207+ if (buf == NULL) return;
1208+
1209+ // Read from http to string "input".
1210+ std::string input;
1211+ char buffer[1024];
1212+ int l=0;
1213+ do {
1214+ l= evbuffer_remove(req->input_buffer, buffer, 1024);
1215+ input.append(buffer, l);
1216+ }while(l);
1217+
1218+ // Schema and table are given in request uri.
1219+ // TODO: If we want to be really NoSQL, we will some day allow to use synonyms like "collection" instead of "table".
1220+ const char *schema;
1221+ const char *table;
1222+ const char *id;
1223+ evhttp_parse_query(evhttp_request_uri(req), req->input_headers);
1224+ schema = (char *)evhttp_find_header(req->input_headers, "schema");
1225+ table = (char *)evhttp_find_header(req->input_headers, "table");
1226+ id = (char *)evhttp_find_header(req->input_headers, "_id");
1227+
1228+ // Set test as default schema
1229+ if ( !strcmp( schema, "") || schema == NULL)
1230+ {
1231+ schema = "test";
1232+ }
1233+
1234+ // Parse "input" into "json_in".
1235+ Json::Value json_in;
1236+ Json::Features json_conf;
1237+ Json::Reader reader(json_conf);
1238+ bool retval = reader.parse(input, json_in);
1239+ if (retval != true) {
1240+ json_out["error_type"]="json error";
1241+ json_out["error_message"]= reader.getFormatedErrorMessages();
1242+ }
1243+ else {
1244+ // It is allowed to specify _id in the uri and leave it out from the json query.
1245+ // In that case we put the value from uri into json_in here.
1246+ // If both are specified, the one existing in json_in wins. (This is still valid, no error.)
1247+ if ( ! json_in["_id"].asBool() )
1248+ {
1249+ if( id ) {
1250+ json_in["_id"] = (Json::Value::UInt) atol(id);
1251+ }
1252+ }
1253+
1254+ // For POST method, we check if table exists.
1255+ // If it doesn't, we automatically CREATE TABLE that matches the structure
1256+ // in the given json document. (This means, your first JSON document must
1257+ // contain all top-level keys you'd like to use.)
1258+ drizzled::Session::shared_ptr _session= drizzled::Session::make_shared(drizzled::plugin::Listen::getNullClient(),
1259+ drizzled::catalog::local());
1260+ drizzled::identifier::user::mptr user_id= identifier::User::make_shared();
1261+ user_id->setUser("");
1262+ _session->setUser(user_id);
1263+ drizzled::Execute execute(*(_session.get()), true);
1264+
1265+ drizzled::sql::ResultSet result_set(1);
1266+ std::string sql="select count(*) from information_schema.tables where table_schema = '";
1267+ sql.append(schema);
1268+ sql.append("' AND table_name = '");
1269+ sql.append(table); sql.append("';");
1270+ /* Execute wraps the SQL to run within a transaction */
1271+ execute.run(sql, result_set);
1272+
1273+ drizzled::sql::Exception exception= result_set.getException();
1274+
1275+ drizzled::error_t err= exception.getErrorCode();
1276+ while(result_set.next())
1277+ {
1278+ if(result_set.getString(0)=="0")
1279+ {
1280+ table_exists = false;
1281+ }
1282+ }
1283+ if(table_exists == false)
1284+ {
1285+ std::string tmp = "CREATE TABLE ";
1286+ tmp.append(schema);
1287+ tmp.append(".");
1288+ tmp.append(table);
1289+ tmp.append(" (_id BIGINT PRIMARY KEY auto_increment,");
1290+ // Iterate over json_in keys
1291+ Json::Value::Members createKeys( json_in.getMemberNames() );
1292+ for ( Json::Value::Members::iterator it = createKeys.begin(); it != createKeys.end(); ++it )
1293+ {
1294+ const std::string &key = *it;
1295+ if(key=="_id") {
1296+ continue;
1297+ }
1298+ tmp.append(key);
1299+ tmp.append(" TEXT");
1300+ if( it !=createKeys.end()-1 && key !="_id")
1301+ {
1302+ tmp.append(",");
1303+ }
1304+ }
1305+ tmp.append(")");
1306+ vector<string> csql;
1307+ csql.clear();
1308+ csql.push_back("COMMIT");
1309+ csql.push_back (tmp);
1310+ sql.clear();
1311+ BOOST_FOREACH(string& it, csql)
1312+ {
1313+ sql.append(it);
1314+ sql.append("; ");
1315+ }
1316+ drizzled::sql::ResultSet createtable_result_set(1);
1317+ execute.run(sql, createtable_result_set);
1318+
1319+ exception= createtable_result_set.getException();
1320+ err= exception.getErrorCode();
1321+ }
1322+ // Now we "parse" the json_in object and build an SQL query
1323+ sql.clear();
1324+ sql.append("REPLACE INTO `");
1325+ sql.append(schema);
1326+ sql.append("`.`");
1327+ sql.append(table);
1328+ sql.append("` SET ");
1329+ // Iterate over json_in keys
1330+ Json::Value::Members keys( json_in.getMemberNames() );
1331+ for ( Json::Value::Members::iterator it = keys.begin(); it != keys.end(); ++it )
1332+ {
1333+ if ( it != keys.begin() )
1334+ {
1335+ sql.append(", ");
1336+ }
1337+ // TODO: Need to do json_in[].type() first and juggle it from there to be safe. See json/value.h
1338+ const std::string &key = *it;
1339+ sql.append(key); sql.append("=");
1340+ Json::StyledWriter writeobject;
1341+ switch ( json_in[key].type() )
1342+ {
1343+ case Json::nullValue:
1344+ sql.append("NULL");
1345+ break;
1346+ case Json::intValue:
1347+ case Json::uintValue:
1348+ case Json::realValue:
1349+ case Json::booleanValue:
1350+ sql.append(json_in[key].asString());
1351+ break;
1352+ case Json::stringValue:
1353+ sql.append("'\"");
1354+ // TODO: MUST be sql quoted!
1355+ sql.append(json_in[key].asString());
1356+ sql.append("\"'");
1357+ break;
1358+ case Json::arrayValue:
1359+ case Json::objectValue:
1360+ sql.append("'");
1361+ sql.append(writeobject.write(json_in[key]));
1362+ sql.append("'");
1363+ break;
1364+ default:
1365+ sql.append("'Error in json_server.cc. This should never happen.'");
1366+ json_out["error_type"]="json error";
1367+ json_out["error_message"]= "json_in object had a value that wasn't of any of the types that we recognize.";
1368+ break;
1369+ }
1370+ sql.append(" ");
1371+ }
1372+ sql.append(";");
1373+ drizzled::sql::ResultSet replace_result_set(1);
1374+
1375+ // Execute wraps the SQL to run within a transaction
1376+ execute.run(sql, replace_result_set);
1377+
1378+ exception= replace_result_set.getException();
1379+
1380+ err= exception.getErrorCode();
1381+
1382+ json_out["sqlstate"]= exception.getSQLState();
1383+
1384+ // TODO: I should be able to return number of rows inserted/updated.
1385+ // TODO: Return last_insert_id();
1386+ if ((err != drizzled::EE_OK) && (err != drizzled::ER_EMPTY_QUERY))
1387+ {
1388+ json_out["error_type"]="sql error";
1389+ json_out["error_message"]= exception.getErrorMessage();
1390+ json_out["error_code"]= exception.getErrorCode();
1391+ json_out["internal_sql_query"]= sql;
1392+ json_out["schema"]= "test";
1393+ }
1394+ json_out["query"]= json_in;
1395+ }
1396+ // Return either the results or an error message, in json.
1397+ Json::StyledWriter writer;
1398+ std::string output= writer.write(json_out);
1399+ evbuffer_add(buf, output.c_str(), output.length());
1400+ evhttp_send_reply(req, HTTP_OK, "OK", buf);
1401+}
1402+
1403+/*
1404+void process_api02_json_put_req(struct evhttp_request *req, void* )
1405+{
1406+ struct evbuffer *buf = evbuffer_new();
1407+ if (buf == NULL) return;
1408+ evhttp_send_reply(req, HTTP_OK, "OK", buf);
1409+}
1410+*/
1411+
1412+void process_api02_json_delete_req(struct evhttp_request *req, void* )
1413+{
1414+ struct evbuffer *buf = evbuffer_new();
1415+ if (buf == NULL) return;
1416+
1417+ Json::Value json_out;
1418+
1419+ std::string input;
1420+ char buffer[1024];
1421+
1422+ // Schema and table are given in request uri.
1423+ // TODO: If we want to be really NoSQL, we will some day allow to use synonyms like "collection" instead of "table".
1424+ // For GET, also the query is in the uri
1425+ const char *schema;
1426+ const char *table;
1427+ const char *query;
1428+ const char *id;
1429+ evhttp_parse_query(evhttp_request_uri(req), req->input_headers);
1430+ schema = (char *)evhttp_find_header(req->input_headers, "schema");
1431+ table = (char *)evhttp_find_header(req->input_headers, "table");
1432+ query = (char *)evhttp_find_header(req->input_headers, "query");
1433+ id = (char *)evhttp_find_header(req->input_headers, "_id");
1434+
1435+ // query can be null if _id was given
1436+ if ( query == NULL || strcmp(query, "") == 0 )
1437+ {
1438+ // Empty JSON object
1439+ query = "{}";
1440+ }
1441+ input.append(query, strlen(query));
1442+
1443+ // Set test as default schema
1444+ if ( !strcmp( schema, "") || schema == NULL)
1445+ {
1446+ schema = "test";
1447+ }
1448+
1449+ // Parse "input" into "json_in".
1450+ Json::Value json_in;
1451+ Json::Features json_conf;
1452+ json_conf.strictMode();
1453+ Json::Reader reader(json_conf);
1454+ bool retval = reader.parse(input, json_in);
1455+ if (retval != true) {
1456+ json_out["error_type"]="json error";
1457+ json_out["error_message"]= reader.getFormatedErrorMessages();
1458+ }
1459+ else {
1460+ // It is allowed to specify _id in the uri and leave it out from the json query.
1461+ // In that case we put the value from uri into json_in here.
1462+ // If both are specified, the one existing in json_in wins. (This is still valid, no error.)
1463+ if ( ! json_in["_id"].asBool() )
1464+ {
1465+ if( id ) {
1466+ json_in["_id"] = (Json::Value::UInt) atol(id);
1467+ }
1468+ }
1469+ // Now we "parse" the json_in object and build an SQL query
1470+ char sqlformat[1024] = "DELETE FROM `%s`.`%s` WHERE _id=%i;";
1471+ sprintf(buffer, sqlformat, schema, table, json_in["_id"].asInt());
1472+ std::string sql = "";
1473+ sql.append(buffer, strlen(buffer));
1474+
1475+ // We have sql string. Use Execute API to run it and convert results back to JSON.
1476+ drizzled::Session::shared_ptr _session= drizzled::Session::make_shared(drizzled::plugin::Listen::getNullClient(),
1477+ drizzled::catalog::local());
1478+ drizzled::identifier::user::mptr user_id= identifier::User::make_shared();
1479+ user_id->setUser("");
1480+ _session->setUser(user_id);
1481+
1482+ drizzled::Execute execute(*(_session.get()), true);
1483+
1484+ drizzled::sql::ResultSet result_set(1);
1485+
1486+ /* Execute wraps the SQL to run within a transaction */
1487+ execute.run(sql, result_set);
1488+ drizzled::sql::Exception exception= result_set.getException();
1489+
1490+ drizzled::error_t err= exception.getErrorCode();
1491+
1492+ json_out["sqlstate"]= exception.getSQLState();
1493+
1494+ if ((err != drizzled::EE_OK) && (err != drizzled::ER_EMPTY_QUERY))
1495+ {
1496+ json_out["error_type"]="sql error";
1497+ json_out["error_message"]= exception.getErrorMessage();
1498+ json_out["error_code"]= exception.getErrorCode();
1499+ json_out["internal_sql_query"]= sql;
1500+ json_out["schema"]= "test";
1501+ }
1502+ json_out["query"]= json_in;
1503+ }
1504+ // Return either the results or an error message, in json.
1505+ Json::StyledWriter writer;
1506+ std::string output= writer.write(json_out);
1507+ evbuffer_add(buf, output.c_str(), output.length());
1508+ evhttp_send_reply(req, HTTP_OK, "OK", buf);
1509+
1510+ }
1511+
1512+
1513 static void shutdown_event(int fd, short, void *arg)
1514 {
1515 struct event_base *base= (struct event_base *)arg;
1516@@ -302,10 +881,25 @@
1517 return false;
1518 }
1519
1520+ // These URLs are available. Bind worker method to each of them.
1521+ // Please group by api version. Also unchanged functions must be copied to next version!
1522 evhttp_set_cb(httpd, "/", process_root_request, NULL);
1523+ // API 0.1
1524 evhttp_set_cb(httpd, "/0.1/version", process_api01_version_req, NULL);
1525 evhttp_set_cb(httpd, "/0.1/sql", process_api01_sql_req, NULL);
1526- evhttp_set_gencb(httpd, process_request, NULL);
1527+ // API 0.2
1528+ evhttp_set_cb(httpd, "/0.2/version", process_api01_version_req, NULL);
1529+ evhttp_set_cb(httpd, "/0.2/sql", process_api01_sql_req, NULL);
1530+ evhttp_set_cb(httpd, "/0.2/json", process_api02_json_req, NULL);
1531+ // API "latest" and also available in top level
1532+ evhttp_set_cb(httpd, "/latest/version", process_api01_version_req, NULL);
1533+ evhttp_set_cb(httpd, "/latest/sql", process_api01_sql_req, NULL);
1534+ evhttp_set_cb(httpd, "/latest/json", process_api02_json_req, NULL);
1535+ evhttp_set_cb(httpd, "/version", process_api01_version_req, NULL);
1536+ evhttp_set_cb(httpd, "/sql", process_api01_sql_req, NULL);
1537+ evhttp_set_cb(httpd, "/json", process_api02_json_req, NULL);
1538+ // Catch all does nothing and returns generic message.
1539+ //evhttp_set_gencb(httpd, process_request, NULL);
1540
1541 event_set(&wakeup_event, wakeup_fd[0], EV_READ | EV_PERSIST, shutdown_event, base);
1542 event_base_set(base, &wakeup_event);
1543@@ -367,7 +961,7 @@
1544 DRIZZLE_VERSION_ID,
1545 "json_server",
1546 "0.1",
1547- "Stewart Smith",
1548+ "Stewart Smith, Henrik Ingo, Mohit Srivastava",
1549 N_("JSON HTTP interface"),
1550 PLUGIN_LICENSE_GPL,
1551 drizzle_plugin::json_server::json_server_init,
1552
1553=== modified file 'plugin/json_server/plugin.ac'
1554--- plugin/json_server/plugin.ac 2011-04-21 01:27:52 +0000
1555+++ plugin/json_server/plugin.ac 2012-05-20 15:31:19 +0000
1556@@ -1,1 +1,9 @@
1557 PANDORA_HAVE_LIBEVENT
1558+
1559+AS_IF([test "x$ac_cv_libevent" = "xno"],
1560+ AC_MSG_WARN([libevent not found: not building json_server.]))
1561+
1562+PANDORA_LIBEVENT_RECENT
1563+
1564+AS_IF([test "$pandora_cv_libevent_recent" = "no"],
1565+ AC_MSG_WARN([Your version of libevent is too old. json_server requires v 2.0 or newer: not building json_server.]))
1566
1567=== modified file 'plugin/json_server/plugin.ini'
1568--- plugin/json_server/plugin.ini 2012-01-14 22:02:03 +0000
1569+++ plugin/json_server/plugin.ini 2012-05-20 15:31:19 +0000
1570@@ -17,5 +17,5 @@
1571 json/json_reader.cpp
1572 json/json_value.cpp
1573 json/json_writer.cpp
1574-build_conditional="x${ac_cv_libevent}" = "xyes" -a "x${ac_cv_libcurl}" = "xyes"
1575+build_conditional="x${ac_cv_libevent}" = "xyes" -a "x$pandora_cv_libevent_recent" = "xyes" -a "x${ac_cv_libcurl}" = "xyes"
1576 ldflags=${LTLIBEVENT}

Subscribers

People subscribed via source and target branches