Posted
almost 11 years
ago
by
A.Bouchez
For pre-Unicode versions of Delphi, the unique way of having UTF-16 native
type is to use the WideString type.
This type, under Windows, matched the BSTR managed
type, as used by OLE and COM components.
In Delphi, WideString implementation calls
... [More]
directly the
corresponding Windows API, and do not use the main Delphi heap manager.
Even if since Vista, this API did have a huge speed-up, it is still in practice
much slower than the regular string type. Problems is not about
UTF-16 encoding, but about the memory allocation, which is shared among
processes, using the Windows global heap, and is much slower than our beloved
FastMM4.
Newer versions of Delphi (since Delphi 2009) feature a refactored
string = UnicodeString type, which relies on
FastMM4 and not the Windows API, and is much faster than
WideString.
Within our mORMot framework,
we by-passed this limitation by using our RawUTF8 type, which is
UTF-8 encoded, so as Unicode ready as the new UnicodeString type,
and pretty fast.
In a recent internal project, we had to use a lot of
WideString instances, to support UTF-16 encoding in Delphi
7/2007, involving a lot of text.
It sounded to be very slow, so we had to do something!
This is where our new SynFastWideString
unit comes in.
Purpose of this unit is to patch the system.pas unit for older
versions of Delphi, so that WideString memory allocation would use
FastMM4 instead of the slow BSTR Windows API.
It will speed up the WideString process a lot, especially when a
lot of content is allocated, since FastMM4 is much more aggressive
than Windows' global heap and the BSTR slow API. It could be more than 50 times
faster, especially when releasing the used memory.
The WideString implementation pattern does NOT feature Copy-On-Write, so is still
slower than the string UnicodeString type as implemented since
Delphi 2009. This is the reason why this unit won't do anything on
Unicode versions of the compiler, since the new string type is to be
preferred there. How to use
Just add the unit at the TOP of your .dpr uses clause, just
after FastMM4 (if you use it, and you should!) i.e. before all other
units used by your program.
It should be initialized before any WideString is
allocated.
Then the patch will be applied at runtime.
Nothing to recompile!
WARNING
USING THIS UNIT MAY BREAK COMPATIBILITY WITH OLE/COM
LIBRARIES !
You won't be able to retrieve and
release WideString/BSTR variables from an OleDB / ADO
database provider, or any COM object.
Do not use this unit if you are calling such external call!
In practice, if you only send some BSTR content to the provider
(e.g. if you use our SynOleDB unit without stored procedure call,
or if you use TWideStringField for most SynDBDataSet
classes), it will work. As far as we tested.
You would have issues if you retrieve a BSTR from the COM
object, or expect the COM object to change the BSTR size, e.g.
with a var WideString parameter or a COM method returning a
WideString.
The following method should work, since it is using a
WideString as input parameter:
type _Catalog = interface(IDispatch) function Create(const ConnectString: WideString): OleVariant; safecall;
But the following won't, since it returns a WideString value,
which will fail to be released by our SynFastWideString unit, since it
is a true BSTR instance allocated by the COM library, and not an
AnsiString allocated via FastMM4:
type _Catalog = interface(IDispatch) function GetObjectOwner(const ObjectName: WideString; ObjectType: OleVariant; ObjectTypeId: OleVariant): WideString; safecall;
In this case, you could use the WideStringFree() procedure, as
defined in the SynFastWideString unit, to release such an
instance.
This unit is for educational purpose only, and/or if you are 100% sure that
your code will stay self-contained, under Delphi 7 or Delphi 2007, and need use
of WideString instead of string AnsiString.
YOU HAVE BEEN WARNED - USE AT YOUR OWN RISK
!
Feedback is welcome, e.g. on Google+
this time! [Less]
|
Posted
almost 11 years
ago
by
A.Bouchez
It is pretty much possible that you would have to maintain and evolve a
legacy project, based on an existing database, with a lot of already written
SQL statements - see Legacy code and
existing projects.
For instance, you would like to use mORMot
... [More]
for new features, and/or
add mobile or HTML clients - see Cross-Platform
clients.
In this case, the ORM advanced features - like ORM Cache or BATCH process, see
BATCH
sequences for adding/updating/deleting records - may conflict with the
legacy code, for the tables which may have to be shared.
Here are some guidelines when working on such a project.
To be exhaustive about your question, we need to consider each ORM CRUD
operation.
We may have to divide them in three kinds: read queries, insertions, and
modifications of existing data. About ORM Retrieve() methods, the ORM cache can be tuned per
table, and you will definitively lack of some cache, but remember :
That you can set a "time out" period for this cache, so that you may still
benefit of it in most cases;
That you have a cache at server level and another at client level, so you
can tune it to be less aggressive on the client, for instance;
That you can tune the ORM cache per ID, so some items which are
not likely to change can still be cached.
About ORM Add() or BatchAdd() methods, when using
the external engine, if any external process is likely to INSERT new rows,
ensure you set the TSQLRestStorageExternal EngineAddUseSelectMaxID
property to TRUE, so that it will compute the next maximum ID by hand.
But it still may be an issue, since the external process may do an INSERT
during the ORM insertion.
So the best is perhaps to NOT use the ORM Add() or
BatchAdd() methods, but rely on dedicated INSERT SQL statement,
e.g. hosted in an interface-based service on the server side.
About ORM Update() Delete() BatchUpdate() BatchDelete()
methods, they sound safe to be used in conjunction with external process
modifying the DB, as soon as you use transactions to let the modifications be
atomic, and won't conflict any concurrent modifications in the legacy code.
Perhaps the safer pattern, when working with external tables which are to be
modified in the background by some legacy code, may be to use server-side
interface-based services - see Client-Server services via
interfaces - for any process involving external tables which may be
modified by another process, with manual SQL, instead of using the ORM "magic".
But it will depend on your business logic, and you will fail to benefit from
the ORM features of the framework.
Nevertheless, introducing Service-Oriented Architecture (SOA) into
your application would be very beneficial: ORM is not mandatory, especially if
you are "fluent" in SQL queries, know how to make them as standard as possible,
and have a lot of legacy code, perhaps with already tuned SQL statements.
Introducing SOA is mandatory to introduce new kind of clients to your
applications, like mobile apps or AJAX modern sites: you could not access
directly the database any more, as you did with your legacy Delphi application,
and RAD DB components.
All new features, involving new tables to store new data, would still benefit
of the mORMot's ORM, and could still be hosted in the very same
external database, shared by your existing code.
Then, you will be able to identify seams - see Legacy code and
existing projects - in your legacy code, and move them to your new
mORMot services, then let your application evolve into a newer
SOA/MVC architecture, without breaking anything, nor starting from
scratch.
Feedback is welcome on our forum, as
usual. [Less]
|
Posted
about 11 years
ago
by
A.Bouchez
If you use our little mORMot framework in any of your project,
commercial or not, please report it to a new
forum thread dedicated for this!
Sounds like if more and more projects are using our little rodent...
The cross-platform clients did increase
... [More]
its popularity, as far as I found out,
in the last days.
Furthermore, if you accept that your application is quoted in our framework
documentation and web site, and/or your logo displayed, please notify it in the
application post or by email to us.
The first listed application - EasyMail7 Client-Server Email Marketing
Solution - is indeed impressive, and shows well the Client-Server and
storage abilities of the framework. Warning: this forum thread is only to notify about applications
created using mORMot.
One post per application!
Please do not post in this thread for reactions or questions about one or
another product.
If you want to discuss, please rather create a new dedicated thread
instead.
Thanks for your feedback! [Less]
|
Posted
about 11 years
ago
by
A.Bouchez
You certainly noticed that WebSocket is the current
trendy flavor for any modern web framework.
But does it scale? Would it replace HTTP/REST?
There is a feature
request ticket about them for mORMot, so here are some thoughts -
matter of debate, of
... [More]
course!
I started all this by answering a StackOverflow
question, in which the actual answers were not accurate enough, to my
opinion.
From my point of view, Websocket - as a protocol -
is some kind of monster.
You start a HTTP stateless connection, then switch to WebSocket
mode which releases the TCP/IP dual-direction layer, then you may switch later
on back to HTTP...
It reminds me some kind of monstrosity, just like encapsulating everything over
HTTP, using XML messages... Just to bypass the security barriers... Just
breaking the OSI layered
model...
It reminds me the fact that our mobile phone data providers do not use
broadcasting for streaming audio and video, but regular Internet HTTP servers,
so the mobile phone data bandwidth is just wasted when a sport event occurs:
every single smart phone has its own connection to the server, and the same
video is transmitted in parallel, saturating the single communication
channel... Smart phones are not so smart, aren't they?
WebSocket sounds like a clever way to circumvent a
limitation...
But why not use a dedicated layer?
I hope HTTP 2.0 would allow
pushing information from the server, as part of the standard... and in one
decade, we probably will see WebSocket as a deprecated
technology.
You have been warned. Do not invest too much in WebSockets..
OK. Back to our existential questions...
First of all, does the WebSocket protocol scale?
Today, any modern single server is able to server millions of clients at
once.
Its HTTP server software has just to be is Event-Driven (IOCP)
oriented (we are not in the old Apache's one connection = one
thread/process equation any more).
Even the HTTP server built in Windows (http.sys - which is used in
mORMot) is IOCP oriented and very efficient (running in kernel
mode).
From this point of view, there won't be a lot of difference at scaling between
WebSocket and a regular HTTP connection. One TCP/IP connection
uses a little resource (much less than a thread), and modern OS are optimized
for handling a lot of concurrent connections: WebSocket and HTTP
are just OSI 7 application layer protocols, inheriting from this TCP/IP
specifications.
But, from experiment, I've seen two main problems with
WebSocket:
It does not support CDN;
It has potential security issues.
About the potential problems of using WebSocket:
1. Consider using a CDN
Today, web scaling involves using Content Delivery
Network (CDN) front ends, not only for static content (html,css,js) but
also your (JSON) application
data.
Of course, you won't put all your data on your CDN cache, but in practice, a
lot of common content won't change often.
I suspect that 80% of your REST resources may be cached...
Even a one minute (or 30 seconds) CDN expiration timeout may be enough
to give your central server a new life, and enhance the application
responsiveness a lot, since CDN can be geographically tuned...
To my knowledge, there is no WebSocket support in CDN yet, and
I suspect it would never be.
The WebSocket protocol is stateful, whereas HTTP is stateless, so
is designed to be cached.
In fact, to make WebSocket CDN-friendly, you may need to switch
to a stateless RESTful approach... which would not be
WebSocket any more.
2. Security issues
WebSocket has several potential security issues, especially
about DOS
attacks.
For illustration about new security vulnerabilities , see
this set of slides and this webkit ticket.
Some quotes from this set of slides:
WebSockets still fall victim to “old” threats;
If you can intercept or inject you can overtake ws:/wss: (MIM
attack);
Malicious content can exhaust browser by grabbing max. allowed number of
WebSocket connections (DOS attacks on client side by browsers!);
Malicious content can create large number of WebSocket connections
to victim WebSocket server (DOS attacks on server);
Just waiting for mistakes to happen (since developers can put anything on
the wire);
wss: means secure transport, not secure app;
Browser support still in flux;
WebSockets solve connection problems, not security problems.
Furthermore, WebSocket avoid any chance of packet inspection
at OSI 7 application layer level.
Application
firewalls, IDS, IPS are pretty
standard nowadays, in any serious security policy.
In fact, WebSocket makes the transmission obfuscated, so may
become a major breach of security leak.
When to use WebSockets?
As a conclusion, I would recommend the following, for any project:
Use WebSocket for client notifications only (with a fallback
mechanism to long-polling - there are plenty of libraries around);
Use RESTful / JSON for all other data, using a CDN or proxies for cache, to
make it scale.
In practice, full WebSocket's applications do not scale well.
Just use WebSocket for what it was designed to: a protocol to
push notifications from the server to the client.
With mORMot, and for all our end-user projects, this is the
direction we are currently entitled to go.
Still privileging a pure stateless approach, which fits so good with our
RESTful model.
Discussion is welcome in our forum, as
usual! [Less]
|
Posted
about 11 years
ago
by
A.Bouchez
The primary purpose of any software Backup is to recover data after
its loss, be it by data deletion or corruption.
Data loss can be a common experience of computer users. A 2008 survey found
that 66% of respondents had lost files on their home PC
... [More]
, as Wikipedia quotes.
As a consequence, for any professional use of data, like in our
mORMot server, a backup policy is mandatory.
We just introduced officially the SQLite3 Backup API to our low-level
SynSQLite3.pas unit, and wrote dedicated methods to make
background backup of a running mORMot server easy and safe, without
any noticeable performance penalty. In all cases, do not forget to perform backups of your SQlite3
database as often as possible (at least several times a day)!
Adding a backup feature on the server side is as simple as running:
Server.DB.BackupBackground('backup.db3',1024,10,nil);
The above line will perform a background live backup of the main
SQLite3 database, by steps of 1024 pages (i.e. it would process 1 MB
per step, since default page size is 1024 bytes), performing a little sleep of
10 milliseconds between each 1 MB copy step, allowing main CRUD / ORM
operations to continue uninterrupted during the backup.
You can even specify an OnProgress: TSQLDatabaseBackupEvent
callback event, to monitor the backup process.
Note that TSQLRestServerDB.Backup or
TSQLRestServerDB.BackupGZ methods are not recommended any more on
a running mORMot database, due to some potential issues with virtual
tables, especially on the Win64 platform. You should definitively use
TSQLDatabase.BackupBackground() instead.
The same backup process can be used e.g. to save an in-memory
SQLite3 database into a SQLite3 file, as such:
if aInMemoryDB.BackupBackground('backup.db3',-1,0,nil) then
aInMemoryDB.BackupBackgroundWaitUntilFinished;
Above code will save the aInMemoryDB database into the
'backup.db3' file.
We wish you a safe and happy backup of your precious objects! [Less]
|
Posted
about 11 years
ago
by
A.Bouchez
Current version of the main framework units target only Win32 and
Win64 systems.
It allows to make easy self-hosting of mORMot servers for local
business applications in any corporation, or pay cheap hosting in the Cloud,
since mORMot CPU and RAM
... [More]
expectations are much lower than a regular
IIS-WCF-MSSQL-.Net stack.
But in a Service-Oriented Architecture (SOA), you would probably need
to create clients for platforms outside the Windows world, especially
mobile devices.
A set of cross-platform client units is therefore available in the
CrossPlatform sub-folder of the source code repository. It allows
writing any client in modern object pascal language, for:
Any version of Delphi, on any platform (Mac OSX, or any
mobile supported devices);
FreePascal Compiler 2.7.1;
Smart Mobile Studio 2.1, to create AJAX or mobile applications
(via PhoneGap, if needed).
This series of articles will introduce you to mORMot's
Cross-Platform abilities:
Units and
platforms;
Generating
the client code wrappers;
Delphi /
FreePascal clients;
Smart Mobile
Studio clients.
Any feedback is welcome in our forum, as
usual! Smart Mobile Studio client samples
In addition to Delphi and FreePascal clients, our
framework is able to access any mORMot server from HTML5 / AJAX rich
client, thanks to Smart Mobile Studio.
Adding two numbers in AJAX
You can find in SQLite3- CrossPlatform ClientsSmartMobileStudio
a simple client for the TServiceCalculator.Add() interface-based
service.
If your Project14ServerHttpWrapper server is running, you can just
point to the supplied wwwhtml file in the sub-folder.
You would then see a web page with a "Server Connect" button, and
if you click on it, you would be able to add two numbers. This a full HTML5 web
application, connecting securely to your mORMot server, which will
work from any desktop browser (on Windows, Mac OS X, or
Linux), or from any mobile device (either iPhone /
iPad / Android / Windows 8 Mobile).
In order to create the application, we just clicked on "download as
file" in the SmartMobileStudio link in the web page, and
copied the generated file in the source folder of a new Smart Mobile
project.
Of course, we did copy the needed SynCrossPlatform*.pas units from
the mORMot source code tree into the Smart library folder, as stated
above. Just ensure you run CopySynCrossPlatformUnits.bat from the
CrossPlatform folder at least once from the latest revision of the
framework source code.
Then, on the form visual editor, we added a BtnConnect button,
then a PanelCompute panel with two edit fields named
EditA and EditB, and two other buttons, named
BtnComputeAsynch and BtnComputeSynch. A
LabelResult label will be used to display the computation result.
The BtnConnect is a toggle which will show or display the
PanelCompute panel, which is hidden by default, depending on the
connection status.
In the Form1.pas unit source code side, we added a reference to
our both SynCrossPlatformREST and mORMotClient units,
and some events to the buttons:
unit Form1;
interface
uses
SmartCL.System, SmartCL.Graphics, SmartCL.Components, SmartCL.Forms,
SmartCL.Fonts, SmartCL.Borders, SmartCL.Application, SmartCL.Controls.Panel,
SmartCL.Controls.Label, SmartCL.Controls.EditBox, SmartCL.Controls.Button,
SynCrossPlatformREST, mORMotClient;
type
TForm1 = class(TW3Form)
procedure BtnComputeSynchClick(Sender: TObject);
procedure BtnComputeAsynchClick(Sender: TObject);
procedure BtnConnectClick(Sender: TObject);
private
{$I 'Form1:intf'}
protected
Client: TSQLRestClientURI;
procedure InitializeForm; override;
procedure InitializeObject; override;
procedure Resize; override;
end;
The BtnConnect event will connect asynchronously to the server,
using 'User' as log-on name, and 'synopse' as
password (those as the framework defaults).
We just use the GetClient() function, as published in our
generated mORMotClient.pas unit:
/// create a TSQLRestClientHTTP instance and connect to the server
// - it will use by default port 888
// - secure connection will be established via TSQLRestServerAuthenticationDefault
// with the supplied credentials
// - request will be asynchronous, and trigger onSuccess or onError event
procedure GetClient(const aServerAddress, aUserName,aPassword: string;
onSuccess, onError: TSQLRestEvent; aServerPort: integer=SERVER_PORT);
It uses two callbacks, the first in case of success, and the second
triggered on failure. On success, we will set the global Client
variable with the TSQLRestClientURI instance just created, then
display the two fields and compute buttons:
procedure TForm1.BtnConnectClick(Sender: TObject);
begin
if Client=nil then
GetClient('127.0.0.1','User','synopse',
lambda (aClient: TSQLRestClientURI)
PanelCompute.Visible := true;
W3Label1.Visible := true;
W3Label2.Visible := true;
LabelConnect.Caption := '';
BtnConnect.Caption := 'Disconnect';
LabelResult.Caption := '';
Client := aClient;
end,
lambda
ShowMessage('Impossible to connect to the server!');
end)
else begin
PanelCompute.Visible := false;
BtnConnect.Caption := 'Server Connect';
Client.Free;
Client := nil;
end;
end;
The GetClient() function expects two callbacks, respectively
onSuccess and onError, which are implemented here
with two SmartPascal lambda blocks.
Now that we are connected to the server, let's do some useful
computation!
As you can see in the mORMotClient.pas generated unit, our
interface-based service can be accessed via a SmartPascal
TServiceCalculator class (and not an interface), with
two variations of each methods: one asynchronous method - e.g.
TServiceCalculator.Add() - expecting success/error callbacks, and
one synchronous (blocking) method - e.g.
TServiceCalculator._Add():
type
/// service accessible via http://localhost:888/root/Calculator
// - this service will run in sicShared mode
// - synchronous and asynchronous methods are available, depending on use case
// - synchronous _*() methods will block the browser execution, so won't be
// appropriate for long process - on error, they may raise EServiceException
TServiceCalculator = class(TServiceClientAbstract)
public
/// will initialize an access to the remote service
constructor Create(aClient: TSQLRestClientURI); override;
procedure Add(n1: integer; n2: integer;
onSuccess: procedure(Result: integer); onError: TSQLRestEvent);
function _Add(const n1: integer; const n2: integer): integer;
end;
We can therefore execute asynchronously the Add() service as
such:
procedure TForm1.BtnComputeAsynchClick(Sender: TObject);
begin
TServiceCalculator.Create(Client).Add(
StrToInt(EditA.Text),StrToInt(EditB.Text),
lambda (res: integer)
LabelResult.Caption := format('Result = %d',[res]);
end,
lambda
ShowMessage('Error calling the method!');
end);
end;
Or execute synchronously the _Add() service:
procedure TForm1.BtnComputeSynchClick(Sender: TObject);
begin
LabelResult.Caption := format('Result = %d',
[TServiceCalculator.Create(Client)._Add(
StrToInt(EditA.Text),StrToInt(EditB.Text))]);
end;
Of course, the synchronous code is much easier to follow and maintain. To be
fair, the SmartPascal lambda syntax is not difficult to
read nor write. In the browser debugger, you can easily set a break point
within any lambda block, and debug your code.
Note that if the server is slow to answer, your whole web application will
be unresponsive, and the browser may even complain about the page, proposing
the kill its process!
As a consequence, simple services may be written in a synchronous manner, but
your serious business code should rather use asynchronous callbacks, just as
with any modern AJAX application.
Thanks to the Smart Linking feature of its compiler, only the used
version of the unit will be converted to JavaScript and included in
the final index.html HTML5 file. So having both synchronous and
asynchronous versions of each method at hand is not an issue.
CRUD/ORM remote access
If the server did have some ORM model, its TSQLRecord classes
will also be part of the mORMotClient.pas generated unit. All
types, even complex record structures, will be marshaled as expected.
For instance, if you run the RegressionTestsServer.dpr server
(available in the same folder), a much more complete unit could be generated
from http://localhost:888/root/wrapper:
type // define some enumeration types, used below
TPeopleSexe = (sFemale, sMale);
TRecordEnum = (reOne, reTwo, reLast);
type // define some record types, used as properties below
TTestCustomJSONArraySimpleArray = record
F: string;
G: array of string;
H: record
H1: integer;
H2: string;
H3: record
H3a: boolean;
H3b: TSQLRawBlob;
end;
end;
I: TDateTime;
J: array of record
J1: byte;
J2: TGUID;
J3: TRecordEnum;
end;
end;
type
/// service accessible via http://localhost:888/root/Calculator
// - this service will run in sicShared mode
// - synchronous and asynchronous methods are available, depending on use case
// - synchronous _*() methods will block the browser execution, so won't be
// appropriate for long process - on error, they may raise EServiceException
TServiceCalculator = class(TServiceClientAbstract)
public
/// will initialize an access to the remote service
constructor Create(aClient: TSQLRestClientURI); override;
procedure Add(n1: integer; n2: integer;
onSuccess: procedure(Result: integer); onError: TSQLRestEvent);
function _Add(const n1: integer; const n2: integer): integer;
procedure ToText(Value: currency; Curr: string; Sexe: TPeopleSexe; Name: string;
onSuccess: procedure(Sexe: TPeopleSexe; Name: string); onError: TSQLRestEvent);
procedure _ToText(const Value: currency; const Curr: RawUTF8; var Sexe: TPeopleSexe; var Name: RawUTF8);
procedure RecordToText(Rec: TTestCustomJSONArraySimpleArray;
onSuccess: procedure(Rec: TTestCustomJSONArraySimpleArray; Result: string); onError: TSQLRestEvent);
function _RecordToText(var Rec: TTestCustomJSONArraySimpleArray): string;
end;
/// map "People" table
TSQLRecordPeople = class(TSQLRecord)
protected
fFirstName: string;
fLastName: string;
fData: TSQLRawBlob;
fYearOfBirth: integer;
fYearOfDeath: word;
fSexe: TPeopleSexe;
fSimple: TTestCustomJSONArraySimpleArray;
// those overriden methods will emulate the needed RTTI
class function ComputeRTTI: TRTTIPropInfos; override;
procedure SetProperty(FieldIndex: integer; const Value: variant); override;
function GetProperty(FieldIndex: integer): variant; override;
public
property FirstName: string read fFirstName write fFirstName;
property LastName: string read fLastName write fLastName;
property Data: TSQLRawBlob read fData write fData;
property YearOfBirth: integer read fYearOfBirth write fYearOfBirth;
property YearOfDeath: word read fYearOfDeath write fYearOfDeath;
property Sexe: TPeopleSexe read fSexe write fSexe;
property Simple: TTestCustomJSONArraySimpleArray read fSimple write fSimple;
end;
In the above code, you can see several methods to the
ICalculator service, some involving the complex
TTestCustomJSONArraySimpleArray record type. The
implementation section of the unit will in fact allow
serialization of such records to/from JSON, even with obfuscated
JavaScript field names, via ComputeRTTI() GetProperty()
and SetProperty().
Some enumerations types are also defined, so will help your
business code be very expressive, thanks to the SmartPascal strong
typing. This is a huge improvement when compared to JavaScript native
weak and dynamic typing.
There is a TSQLRecordPeople class generated, which will map the
following Delphi class type, as defined in the
PeopleServer.pas unit:
TSQLRecordPeople = class(TSQLRecord)
protected
fData: TSQLRawBlob;
fFirstName: RawUTF8;
fLastName: RawUTF8;
fYearOfBirth: integer;
fYearOfDeath: word;
fSexe: TPeopleSexe;
fSimple: TTestCustomJSONArraySimpleArray;
public
class procedure InternalRegisterCustomProperties(Props: TSQLRecordProperties); override;
published
property FirstName: RawUTF8 read fFirstName write fFirstName;
property LastName: RawUTF8 read fLastName write fLastName;
property Data: TSQLRawBlob read fData write fData;
property YearOfBirth: integer read fYearOfBirth write fYearOfBirth;
property YearOfDeath: word read fYearOfDeath write fYearOfDeath;
property Sexe: TPeopleSexe read fSexe write fSexe;
public
property Simple: TTestCustomJSONArraySimpleArray read fSimple;
end;
Here, a complex TTestCustomJSONArraySimpleArray record field
has been published, thanks to a manual
InternalRegisterCustomProperties() registration, as we already
stated above.
You can see that types like RawUTF8 were mapped to the standard
SmartPascal string type, as expected, when converted to
the mORMotClient.pas generated unit.
Your AJAX client can then access to this TSQLRecordPeople
content easily, via standard CRUD operations.
See the SQLite3- SmartMobileStudio Client sample, for instance the
following line:
people := new TSQLRecordPeople;
for i := 1 to 200 do begin
assert(client.Retrieve(i,people));
assert(people.ID=i);
assert(people.FirstName='First' IntToStr(i));
assert(people.LastName='Last' IntToStr(i));
assert(people.YearOfBirth=id 1800);
assert(people.YearOfDeath=id 1825);
end;
Here, the client variable is a TSQLRestClientURI
instance, as returned by the GetClient() onSuccess callback
generated in mORMotClient.pas.
You have Add() Delete() Update() FillPrepare()
CreateAndFillPrepare() and Batch*() methods available,
ready to safely access your data from your AJAX client.
If you update your data model on the server, just re-generate your
mORMotClient.pas unit from
http://localhost:888/root/wrapper, then rebuild your Smart
Mobile Studio project to reflect all changes made to your ORM data model,
or your SOA available services.
Thanks to the SmartPascal strong typing, any breaking change of the
server expectations would immediately be reported at compilation, and not at
runtime, as it would with regular JavaScript clients. [Less]
|
Posted
about 11 years
ago
by
A.Bouchez
Current version of the main framework units target only Win32 and
Win64 systems.
It allows to make easy self-hosting of mORMot servers for local
business applications in any corporation, or pay cheap hosting in the Cloud,
since mORMot CPU and RAM
... [More]
expectations are much lower than a regular
IIS-WCF-MSSQL-.Net stack.
But in a Service-Oriented Architecture (SOA), you would probably need
to create clients for platforms outside the Windows world, especially
mobile devices.
A set of cross-platform client units is therefore available in the
CrossPlatform sub-folder of the source code repository. It allows
writing any client in modern object pascal language, for:
Any version of Delphi, on any platform (Mac OSX, or any
mobile supported devices);
FreePascal Compiler 2.7.1;
Smart Mobile Studio 2.1, to create AJAX or mobile applications
(via PhoneGap, if needed).
This series of articles will introduce you to mORMot's
Cross-Platform abilities:
Units and
platforms;
Generating
the client code wrappers;
Delphi /
FreePascal clients;
Smart Mobile
Studio clients.
Any feedback is welcome in our forum, as
usual! Delphi / FreePascal client samples
The "27 - CrossPlatform ClientsRegressionTests.dpr" sample
creates a mORMot server with its own ORM data model, containing a
TSQLRecordPeople class, and a set of interface-based SOA services,
some including complex types like a record.
Then this sample uses a generated mORMotClient.pas, retrieved
from the "download as file" link of the
CrossPlatform template above.
Its set of regression tests (written using a small cross-platform
TSynTest unit test class) will then perform remote ORM and SOA
calls to the PeopleServer embedded instance, over all supported
authentication schemes - see Authentication:
Cross Platform Units for mORMot
---------------------------------
1. Running "Iso8601DateTime"
30003 tests passed in 00:00:018
2. Running "Base64Encoding"
304 tests passed in 00:00:000
3. Running "JSON"
18628 tests passed in 00:00:056
4. Running "Model"
1013 tests passed in 00:00:003
5. Running "Cryptography"
4 tests passed in 00:00:000
Tests failed: 0 / 49952
Time elapsed: 00:00:080
Cross Platform Client for mORMot without authentication
---------------------------------------------------------
1. Running "Connection"
2 tests passed in 00:00:010
2. Running "ORM"
4549 tests passed in 00:00:160
3. Running "ORMBatch"
4564 tests passed in 00:00:097
4. Running "Services"
26253 tests passed in 00:00:302
5. Running "CleanUp"
1 tests passed in 00:00:000
Tests failed: 0 / 35369
Time elapsed: 00:00:574
Cross Platform Client for mORMot using TSQLRestServerAuthenticationNone
-------------------------------------------------------------------------
...
Cross Platform Client for mORMot using TSQLRestServerAuthenticationDefault
----------------------------------------------------------------------------
...
The generated mORMotClient.pas unit is used for all
"Cross Platform Client" tests above, covering both ORM and SOA
features of the framework.
Connection to the server
You could manually connect to a mORMot server as such:
var Model: TSQLModel;
Client: TSQLRestClientHTTP;
...
Model := TSQLModel.Create([TSQLAuthUser,TSQLAuthGroup,TSQLRecordPeople]);
Client := TSQLRestClientHTTP.Create('localhost',SERVER_PORT,Model);
if not Client.Connect then
raise Exception.Create('Impossible to connect to the server');
if Client.ServerTimeStamp=0 then
raise Exception.Create('Incorrect server');
if not Client.SetUser(TSQLRestAuthenticationDefault,'User','synopse') then
raise Exception.Create('Impossible to authenticate to the server');
...
Or you may use the GetClient() function generated in
mORMotClient.pas:
/// create a TSQLRestClientHTTP instance and connect to the server
// - it will use by default port 888
// - secure connection will be established via TSQLRestServerAuthenticationDefault
// with the supplied credentials - on connection or authentication error,
// this function will raise a corresponding exception
function GetClient(const aServerAddress, aUserName,aPassword: string;
aServerPort: integer=SERVER_PORT): TSQLRestClientHTTP;
Which could be used as such:
var Client: TSQLRestClientHTTP;
...
Client := GetClient('localhost','User','synopse')
The data model and the expected authentication scheme were included in the
GetClient() function, which will raise the expected
ERestException in case of any connection or authentication
issue.
CRUD/ORM remote access
Thanks to SynCrossPlatform* units, you could easily perform any
remote ORM operation on your mORMot server, with the usual
TSQLRest CRUD methods.
For instance, the RegressionTests.dpr sample performs the
following operations
fClient.CallBackGet('DropTable',[],Call,TSQLRecordPeople); // call of method-based service
check(Call.OutStatus=HTML_SUCCESS);
people := TSQLRecordPeople.Create; // create a record ORM
try
for i := 1 to 200 do begin
people.FirstName := 'First' IntToStr(i);
people.LastName := 'Last' IntToStr(i);
people.YearOfBirth := i 1800;
people.YearOfDeath := i 1825;
people.Sexe := TPeopleSexe(i and 1);
check(Client.Add(people,true)=i); // add one record
end;
finally
people.Free;
end;
...
people := TSQLRecordPeople.CreateAndFillPrepare(fClient,'',
'yearofbirth=?',[1900]); // parameterized query returning one or several rows
try
n := 0;
while people.FillOne do begin
inc(n);
check(people.ID=100);
check(people.FirstName='First100');
check(people.LastName='Last100');
check(people.YearOfBirth=1900);
check(people.YearOfDeath=1925);
end;
check(n=1); // we expected only one record here
finally
people.Free;
end;
for i := 1 to 200 do
if i and 15=0 then
fClient.Delete(TSQLRecordPeople,i) else // record deletion
if i mod 82=0 then begin
people := TSQLRecordPeople.Create;
try
id := i 1;
people.ID := i;
people.YearOfBirth := id 1800;
people.YearOfDeath := id 1825;
check(fClient.Update(people,'YEarOFBIRTH,YEarOfDeath')); // record modification
finally
people.Free;
end;
end;
for i := 1 to 200 do begin
people := TSQLRecordPeople.Create(fClient,i); // retrieve one instance from ID
try
if i and 15=0 then // was deleted
Check(people.ID=0) else begin
if i mod 82=0 then
id := i 1 else // was modified
id := i;
Check(people.ID=i);
Check(people.FirstName='First' IntToStr(i));
Check(people.LastName='Last' IntToStr(i));
Check(people.YearOfBirth=id 1800);
Check(people.YearOfDeath=id 1825);
Check(ord(people.Sexe)=i and 1);
end;
finally
people.Free;
end;
end;
As we already stated, BATCH mode is also supported, with the classic
mORMot syntax:
...
res: TIntegerDynArray;
...
fClient.BatchStart(TSQLRecordPeople);
people := TSQLRecordPeople.Create;
try
for i := 1 to 200 do begin
people.FirstName := 'First' IntToStr(i);
people.LastName := 'Last' IntToStr(i);
people.YearOfBirth := i 1800;
people.YearOfDeath := i 1825;
fClient.BatchAdd(people,true);
end;
finally
people.Free;
end;
fClient.fBatchSend(res)=HTML_SUCCESS);
check(length(res)=200);
for i := 1 to 200 do
check(res[i-1]=i); // server returned the IDs of the newly created records
Those BatchAdd / BatchDelete /
BatchUpdate methods of TSQLRest have the benefit to
introduce at client level:
Much higher performance, especially on multi-insertion or multi-update of
data;
Transactional support: TSQLRest.BatchStart() has an optional
AutomaticTransactionPerRow parameter, set to 10000 by
default, which will create a server-side transaction during the write process,
and an ACID rollback in case of any failure.
You can note that all above code has exactly the same structure and methods
than standard mORMot clients.
The generated mORMotClient.pas unit contains all needed
TSQLRecord types, and its used properties, including enumerations
or complex records. The only dependency of this unit are
SynCrossPlatform* units, so would be perfectly cross-platform
(whereas our main SynCommons.pas and mORMot.pas units
do target only Win32 and Win64).
As a result, you are able to share server and client code between a
Windows project and any supported platform, even AJAX (see "Smart Mobile
Studio client samples" below). A shared unique code base would eventually
reduce both implementation and debugging time, which is essential to unleash
your business code potential and maximize your ROI.
Service consumption
The ultimate goal of the mORMot framework is to publish your
business via a Service-Oriented Architecture (SOA).
As a consequence, those services should be made available from any kind of
device or platform, even outside the Windows world. The server is able
to generate client wrappers code, which could be used to consume any
Client-Server services via interfaces using any supported
authentication scheme - see Authentication.
Here is an extract of the mORMotClient.pas unit as generated
for the RegressionTests.dpr sample:
type
/// service implemented by TServiceCalculator
// - you can access this service as such:
// !var aCalculator: ICalculator;
// !begin
// ! aCalculator := TCalculator.Create(aClient);
// ! // now you can use aCalculator methods
// !...
ICalculator = interface(IServiceAbstract)
['{9A60C8ED-CEB2-4E09-87D4-4A16F496E5FE}']
function Add(const n1: integer; const n2: integer): integer;
procedure ToText(const Value: currency; const Curr: string; var Sexe: TPeopleSexe; var Name: string);
function RecordToText(var Rec: TTestCustomJSONArraySimpleArray): string;
end;
/// implements ICalculator from http://localhost:888/root/Calculator
// - this service will run in sicShared mode
TServiceCalculator = class(TServiceClientAbstract,ICalculator)
public
constructor Create(aClient: TSQLRestClientURI); override;
function Add(const n1: integer; const n2: integer): integer;
procedure ToText(const Value: currency; const Curr: string; var Sexe: TPeopleSexe; var Name: string);
function RecordToText(var Rec: TTestCustomJSONArraySimpleArray): string;
end;
As you can see, a dedicated class has been generated to consume the
server-side ICalculator interface-based service, in its own
ICalculator client-side type.
It is able to handle complex types, like enumerations (e.g.
TPeopleSexe) and records (e.g.
TTestCustomJSONArraySimpleArray), which are also defined in the
very same mORMotClient.pas unit.
You can note that the RawUTF8 type has been changed into the
standard Delphi / FreePascal string type, since
it is the native type used by our SynCrossPlatformJSON.pas unit
for all its JSON marshaling. Of course, under latest version of Delphi
and FreePascal, this kind of content may be Unicode encoded (either as
UTF-16 for the string = UnicodeString Delphi
type, or as UTF-8 for the FreePascal / Lazarus
string type).
The supplied regression tests show how to use remotely those services:
var calc: ICalculator;
i,j: integer;
sex: TPeopleSexe;
name: string;
...
calc := TServiceCalculator.Create(fClient);
check(calc.InstanceImplementation=sicShared);
check(calc.ServiceName='Calculator');
for i := 1 to 200 do
check(calc.Add(i,i 1)=i*2 1);
for i := 1 to 200 do begin
sex := TPeopleSexe(i and 1);
name := 'Smith';
calc.ToText(i,'$',sex,name);
check(sex=sFemale);
check(name=format('$ %d for %s Smith',[i,SEX_TEXT[i and 1]]));
end;
...
As with regular mORMot client code, a
TServiceCalculator instance is created and is assigned to a
ICalculator local variable. As such, no try ... finally
Calc.Free end block is mandatory here, to avoid any memory leak: the
compiler will create such an hidden block for the Calc:
ICalculator variable scope.
The service-side contract of the ICalculator signature is
retrieved and checked within TServiceCalculator.Create, and would
raise an ERestException if it does not match the contract
identified in mORMotClient.pas.
The cross-platform clients are able to manage the service instance
life-time, especially the sicPerClient mode. In this case, an
implementation class instance will be created on the server for each client,
until the corresponding interface instance will released (i.e. out
of scope or assigned to nil), which will release the server-side
instance - just like with a regular mORMot client code.
Note that all process here is executed synchronously, i.e. in
blocking mode. It is up to you to ensure that your application is able to still
be responsive, even if the server does a lot of process, so may be late to
answer. A dedicated thread may help in this case. [Less]
|
Posted
about 11 years
ago
by
A.Bouchez
Current version of the main framework units target only Win32 and
Win64 systems.
It allows to make easy self-hosting of mORMot servers for local
business applications in any corporation, or pay cheap hosting in the Cloud,
since mORMot CPU and RAM
... [More]
expectations are much lower than a regular
IIS-WCF-MSSQL-.Net stack.
But in a Service-Oriented Architecture (SOA), you would probably need
to create clients for platforms outside the Windows world, especially
mobile devices.
A set of cross-platform client units is therefore available in the
CrossPlatform sub-folder of the source code repository. It allows
writing any client in modern object pascal language, for:
Any version of Delphi, on any platform (Mac OSX, or any
mobile supported devices);
FreePascal Compiler 2.7.1;
Smart Mobile Studio 2.1, to create AJAX or mobile applications
(via PhoneGap, if needed).
This series of articles will introduce you to mORMot's
Cross-Platform abilities:
Units and
platforms;
Generating
the client code wrappers;
Delphi /
FreePascal clients;
Smart Mobile
Studio clients.
Any feedback is welcome in our forum, as
usual! Generating client wrappers
Even if it feasible to write the client code by hand, your mORMot
server is able to create the source code needed for client access, via a
dedicated method-based service, and set of Mustache-based templates -
see Mustache
template engine.
The following templates are available in the
CrossPlatformtemplates folder:
Unit Name
Compiler Target
CrossPlatform.pas.mustache
Delphi / FPC SynCrossPlatform* units
Delphi.pas.mustache
Delphi Win32/Win64 mORMot units
SmartMobileStudio.pas.mustache
Smart Mobile Studio 2.1
In the future, other wrappers may be added. And you can write your own,
which could be included within the framework source! Your input is warmly
welcome, especially if you want to write a template for Java or
C# client. The generated data context already contains the data
types corresponding to those compilers: e.g. a mORMot's
RawUTF8 field or parameter could be identified as
"typeCS":"string" or "typeJava":"String" in addition
to "typeDelphi":"RawUTF8" and
"typePascal":"string".
Publishing the code generator
By default, and for security reasons, the code generation is not embedded to
your mORMot RESTful server. In fact, the
mORMotWrapper.pas unit will link both mORMot.pas and
SynMustache.pas units, and use Mustache templates to
generate code for a given TSQLRestServer instance.
We will start from the interface-based service Sample code as
defined in the
"SQLite3- Interface based services" folder.
After some minor modifications, we copied the server source code into
"SQLite3- CrossPlatform
ClientsProject14ServerHttpWrapper.dpr":
program Project14ServerHttpWrapper;
{$APPTYPE CONSOLE}
uses
SysUtils,
Classes,
SynCommons,
mORMot,
mORMotHttpServer,
mORMotWrappers,
Project14Interface in '..\14 - Interface based services\Project14Interface.pas';
type
TServiceCalculator = class(TInterfacedObject, ICalculator)
public
function Add(n1,n2: integer): integer;
end;
function TServiceCalculator.Add(n1, n2: integer): integer;
begin
result := n1+n2;
end;
var
aModel: TSQLModel;
aServer: TSQLRestServer;
aHTTPServer: TSQLHttpServer;
begin
// create a Data Model
aModel := TSQLModel.Create([],ROOT_NAME);
try
// initialize a TObjectList-based database engine
aServer := TSQLRestServerFullMemory.Create(aModel,'test.json',false,true);
try
// add the http://localhost:888/root/wrapper code generation web page
AddToServerWrapperMethod(aServer,
['..\..\..\CrossPlatform\templates','..\..\..\..\CrossPlatform\templates']);
// register our ICalculator service on the server side
aServer.ServiceRegister(TServiceCalculator,[TypeInfo(ICalculator)],sicShared);
// launch the HTTP server
aHTTPServer := TSQLHttpServer.Create(PORT_NAME,[aServer],'+',useHttpApiRegisteringURI);
try
aHTTPServer.AccessControlAllowOrigin := '*'; // for AJAX requests to work
writeln(#10'Background server is running.');
writeln('You can test http://localhost:',PORT_NAME,'/wrapper');
writeln(#10'Press [Enter] to close the server.'#10);
readln;
finally
aHTTPServer.Free;
end;
finally
aServer.Free;
end;
finally
aModel.Free;
end;
end.
As you can see, we just added a reference to the mORMotWrappers
unit, and a call to AddToServerWrapperMethod() in order to publish
the available code generators.
Now, if you run the Project14ServerHttpWrapper server, and
point your favorite browser to http://localhost:888/root/wrapper
you will see the following page:
Client Wrappers
Available Templates:
* CrossPlatformmORMotClient.pas -
download as file - see as text - see
template
* DelphimORMotClient.pas - download as
file - see as text - see template
* SmartMobileStudiomORMotClient.pas -
download as file - see as text - see
template
You can also retrieve the corresponding template context.
Each of the *.mustache template available in the specified
folder is listed here. Links above will allow downloading a client source code
unit, or displaying it as text in the browser. The template can also be
displayed un-rendered, for reference. As true Mustache templates, the
source code files are generated from a data context, which can be
displayed, as JSON, from the "template context" link. It may help
you when debugging your own templates. Note that if you modify and save a
.mustache template file, just re-load the "see as text"
browser page and your modification is taken in account immediately (you do not
need to restart the server).
Generated source code will follow the template name, and here will always be
downloaded as mORMotClient.pas. Of course, you can change the unit
name for your end-user application. It could be even mandatory if the same
client would access to several mORMot servers at once, which could be
the case in a Service-Oriented Architecture (SOA) project.
Just ensure that you will never change the mORMotClient.pas
generated content by hand. If necessary, you can create and customize your own
Mustache template, to be used for your exact purpose. By design, such
automated code generation will require to re-create the client unit each time
the server ORM or SOA structure is modified. In fact, as stated in the
mORMotClient.pas comment, any manual modification of this file may
be lost after regeneration. You have been warned!
If you feel that the current templates have some issues or need some
enhancements, you are very welcome to send us your change requests on our
forums. Once you are used at it, Mustache templates are fairly easy to
work with. Similarly, if you find out that some information is missing in the
generated data context, e.g. for a new platform or language, we would
be pleased to enhance the official mORMotWrapper.pas process.
Visit our source code repository
There is nothing better than some real files.
You can take a look at the following files in our source code
repository:
The .mustache
template files;
The SynCrossPlatform*.pas
units;
A generated
mORMotClient.pas unit for Delphi / FPC -
generated from
RegressionTestsServer.dpr;
The same
mORMotClient.pas unit for Smart Mobile Studio - also
generated from RegressionTestsServer.dpr;
Another simplier
mORMotClient.pas unit for Smart Mobile Studio - generated
from
Project14ServerHttpWrapper.dpr.
Enjoy! [Less]
|
Posted
about 11 years
ago
by
A.Bouchez
Current version of the main framework units target only Win32 and
Win64 systems.
It allows to make easy self-hosting of mORMot servers for local
business applications in any corporation, or pay cheap hosting in the Cloud,
since mORMot CPU and RAM
... [More]
expectations are much lower than a regular
IIS-WCF-MSSQL-.Net stack.
But in a Service-Oriented Architecture (SOA), you would probably need
to create clients for platforms outside the Windows world, especially
mobile devices.
A set of cross-platform client units is therefore available in the
CrossPlatform sub-folder of the source code repository. It allows
writing any client in modern object pascal language, for:
Any version of Delphi, on any platform (Mac OSX, or any
mobile supported devices);
FreePascal Compiler 2.7.1;
Smart Mobile Studio 2.1, to create AJAX or mobile applications
(via PhoneGap, if needed).
This series of articles will introduce you to mORMot's
Cross-Platform abilities:
Units and
platforms;
Generating
the client code wrappers;
Delphi /
FreePascal clients;
Smart Mobile
Studio clients.
Any feedback is welcome in our forum, as
usual! Involved units
The units are the following:
Unit Name
Description
SynCrossPlatformREST.pas
Main unit, implementing secured ORM and SOA RESTful client
SynCrossPlatformCrypto.pas
SHA-256 and crc32 algorithms, used for authentication
SynCrossPlatformJSON.pas
Optimized JSON process (not used by Smart)
SynCrossPlatformSpecific.pas
System-specific functions, e.g. HTTP clients
This set of units will provide a solid and shared ground for the any kind of
clients:
Connection to a mORMot server via HTTP, with full REST
support;
Support of weak or default authentication to secure the transfer - see
Authentication;
Definition of the TSQLRecord class, using RTTI when available
on Delphi or FreePascal, and generated code for Smart
Mobile Studio;
Remote CRUD operations, via JSON and REST, with a
TSQLRestClientURI class, with the same methods as with the
mORMot.pas framework unit;
Optimized TSQLTableJSON class to handle a JSON result table,
as returned by mORMot's REST server ORM - see JSON (not) expanded
layouts;
Batch process - see BATCH sequences for adding/updating/deleting
records - for transactional and high-speed writes;
Client-Server services via methods with parameters
marshaling;
Client-Server services via interfaces with parameters marshaling
and instance-life time;
Mapping of most supported field types, including e.g. ISO 8601 date/time
encoding, BLOBs and TModTime/TCreateTime - see
TSQLRecord fields definition in the SAD 1.18 pdf;
Complex record types are also exported and consumed via JSON,
on all platforms (for both ORM and SOA methods);
Some cross-platform low-level functions and types definitions, to help
share as much code as possible between your projects.
In the future, C# or Java clients may be written.
The CrossPlatform sub-folder code could be used as reference, to
write minimal and efficient clients on any platform. Our REST model is pretty
straightforward and standard, and use of JSON tends to leverage a lot of
potential marshaling issues which may occur with XML or binary formats.
In practice, a code generator embedded in the mORMot server can be
used to create the client wrappers, using the Mustache template engine
included on the server side. With a click, you can generate and download a
client source file for any supported platform. A set of .mustache
templates is available, and can be customized or extended to support any new
platform: any help is welcome, especially for targeting Java or C# clients.
Available client platforms
Delphi FMX / FreePascal FCL cross-platform support
Latest versions of Delphi include the FireMonkey (FMX)
framework, able to deliver multi-device, true native applications for
Windows, Mac OSX, Android and iOS
(iPhone/iPad).
Our SynCrossPlatform* units are able to easily create clients for
those platforms.
Similarly, these units can be compiled with FreePascal, so that any
mORMot server could be consumed from the numerous supported platforms
of this compiler.
In order to use those units, ensure in your IDE that the
CrossPlatform sub-folder of the mORMot source code
repository is defined in your Library Search Path.
Cross-platform JSON
We developed our own cross-platform JSON process unit in
SynCrossPlatformJSON.pas, shared with Delphi and
FreePascal.
In fact, it appears to be easier to use (since it is variant-based
and with late-binding abilities) and run much faster than the official
DBXJSON.pas unit shipped with latest versions of Delphi,
as stated by the "25 - JSON performance" sample:
2.2. Table content:
- Synopse crossplatform: 41,135 assertions passed 20.56ms 400,048/s 1.9 MB
- DBXJSON: 41,136 assertions passed 240.84ms 34,159/s 9.9 MB
Our TSQLTableJSON class is more than 10 times faster than
standard DBXJSON unit, when processing a list of results as
returned by a mORMot server.
The latest value on each line above is the memory consumption. It should be of
high interest on mobile platforms, where memory allocation tends to be much
slower and sensitive than on Windows (where FastMM4 memory manager
does wonders). Our unit consumes 5 times less memory than the RTL's
version.
We did not include XSuperObject here, which is cross-platform, but
performs even worse than DBXJSON in terms of speed. Other
libraries - as SuperObject or dwsJSON - are not
cross-platform.
See
http://blog.synopse.info/post/json-benchmark-delphi-mormot-superobject-dwsjson-dbxjson
for details about this comparison.
A special mention is due to dwsJSON, which performs very well, but
only on Windows, and is slower than mORMot's implementation:
- Synopse ORM loop: 41,135 assertions passed 6.18ms 1,330,153/s 1.1 MB
- Synopse ORM list: 41,135 assertions passed 6.47ms 1,270,775/s 952 KB
- Synopse crossplatform: 41,135 assertions passed 20.56ms 400,048/s 1.9 MB
- Super object properties: 41,136 assertions passed 2.20s 3,739/s 6.3 MB
- dwsJSON: 41,136 assertions passed 32.05ms 256,628/s 4.7 MB
- DBXJSON: 41,136 assertions passed 240.84ms 34,159/s 9.9 MB
The "Synopse ORM" lines stand for the
TSQLTableJSON class as implemented in mORMot.pas. It
uses our optimized UTF-8 functions and classes, in-place escaping together with
our RawUTF8 custom string type, so that it is 3 times faster than
our cross-platform units, and 40 times than DBXJSON, using much
less memory. Some tricks used by Synopse ORM rely on pointers and
are not compatible with the NextGen compiler or the official
Delphi road-map, so the Synopse crossplatform uses
diverse algorithm, but offers still pretty good performance.
This unit features a TJSONVariantData custom variant type,
similar to TDocVariant custom variant type, available in the main
mORMot framework.
It allows writing such nice and readable code, with late-binding:
var doc: variant;
json,json2: string;
...
doc := JSONVariant('{"test":1234,"name":"Joh\\"n\\r","zero":0.0}');
assert(doc.test=1234);
assert(doc.name='Joh"n'#13);
assert(doc.name2=null);
assert(doc.zero=0);
json := doc; // conversion to JSON text when assigned to a string variable
assert(json='{"test":1234,"name":"Joh\\"n\\r","zero":0}');
doc.name2 := 3.1415926;
doc.name := 'John';
json := doc;
assert(json='{"test":1234,"name":"John","zero":0,"name2":3.1415926}');
The unit is also able to serialize any TPersistent class, i.e.
all published properties could be written or read from a JSON object
representation. It also handles nested objects, stored as
TCollection.
See for instance in the SynCrossPlatformTests unit:
type
TMainNested = class(TCollectionItem)
private
fNumber: double;
fIdent: RawUTF8;
published
property Ident: RawUTF8 read fIdent write fIdent;
property Number: double read fNumber write fNumber;
end;
TMain = class(TPersistent)
private
fName: RawUTF8;
fNested: TCollection;
fList: TStringList;
public
constructor Create;
destructor Destroy; override;
published
property Name: RawUTF8 read fName write fName;
property Nested: TCollection read fNested;
property List: TStringList read fList;
end;
obj1 := TMain.Create;
obj2 := TMain.Create;
...
obj1.Name := IntToStr(i);
item := obj1.Nested.Add as TMainNested;
item.Ident := obj1.Name;
item.Number := i/2;
obj1.list.Add(obj1.Name);
json := ObjectToJSON(obj1);
if i=1 then
assert(json='{"Name":"1","Nested":[{"Ident":"1","Number":0.5}],"List":["1"]}');
JSONToObject(obj2,json);
assert(obj2.Nested.Count=i);
json2 := ObjectToJSON(obj2);
assert(json2=json);
...
Of course, this serialization feature is used for the
TSQLRecord ORM class.
Due to lack of RTTI, record serialization is supported via some
functions generated by the server with the code wrappers.
Delphi OSX and NextGen
In order to be compliant with the NextGen revision,
our SynCrossPlatform* units follow the expectations of this new
family of cross-compilers, which targets Android and
iOS.
In particular, we rely only on the string type for text process
and storage, even at JSON level, and we tried to make object allocation
ARC-compatible. Some types have been defined, e.g. THttpBody,
TUTF8Buffer or AnsiChar, to ensure that our units
would compile on all supported platforms.
On Delphi, the Indy library is used for HTTP requests. It
is cross-platform by nature, so should work on any supported system. For SSL
support with iOS and Android clients, please follow
instructions at http://blog.marcocantu.com/blog/using_ssl_delphi_ios.html
you may also download the needed libcrypto.a and
libssl.a files from http://indy.fulgan.com/SSL/OpenSSLStaticLibs.7z
Feedback is needed for the mobile targets, via FMX.
In fact, we rely for our own projects on Smart Mobile Studio for our
mobile applications, so the Synopse team did not test Delphi
NextGen platforms (i.e. iOS and Android) as deep as
other systems. Your input would be very valuable and welcome, here!
FreePascal clients
SynCrossPlatform* units support the FreePascal
Compiler, in its 2.7.1 revision.
Most of the code is shared with Delphi, including RTTI support and all
supported types.
Some restrictions apply, though.
Due to a bug in FreePascal implementation of variant
late binding, the following code won't work as expected:
doc.name2 := 3.1415926;
doc.name := 'John';
Under FreePascal, you have to write:
TJSONVariantData(doc)['name2'] := 3.1415926;
TJSONVariantData(doc)['name'] := 'John';
In fact, the way late-binding properties are implemented in the
FreePascal RTL forbid to modify the content of the associated
variant. A private copy of the variant is made, which
is not only slower, but disallows modification of its stored value.
Any feedback and help from the FreePascal maintainers may be
welcome!
As a result, direct access to TJSONVariantData instances, and
not a variant variable, would be faster and less error-prone when
using this compiler, until the issue is fixed.
Another issue with the 2.7.1 revision is how the new string
type is implemented.
In fact, if you use a string variable containing an UTF-8 encoded text, then
the following line would reset the result code page to the system code
page:
function StringToJSON(const Text: string): string;
...
result := '"'+copy(Text,1,j-1); // here FPC 2.7.1 erases UTF-8 encoding
...
It sounds like if '"' will force the code page of
result to be not an UTF-8 content.
With Delphi, this kind of statements work as expected, even for
AnsiString values, and '"' constant is handled as
RawByteString. We were not able to find an easy and safe
workaround for FPC yet. Input is welcome in this area, from any expert!
You have to take care of this limitation, if you target the Windows
operating system with FPC (and Lazarus). Under other systems, the
default code page is likely to be UTF-8, so in this case our
SynCrossPlatform* units will work as expected.
We found out the FreePascal compiler to work very well, and result
in small and fast executables. For most common work, timing is comparable with
Delphi. The memory manager is less optimized than FastMM4 for
rough simple threaded tests, but is cross-platform and much more efficient in
multi-thread mode: in fact, it has no giant lock, as FastMM4
suffers.
Smart Mobile Studio support
Smart Mobile Studio - see http://www.smartmobilestudio.com - is a
complete RAD environment for writing cutting edge HTML5 mobile applications. It
ships with a fully fledged compiler capable of compiling Object Pascal
(in a modern dialect call SmartPascal) into highly optimized and raw
JavaScript.
There are several solutions able to compile to JavaScript.
In fact, we can find several families of compilers:
JavaScript super-sets, adding optional strong typing, and
classes, close to the ECMAScript Sixth Edition: the current main
language in this category is certainly TypeScript, designed by Anders
Hejlsberg (father of both the Delphi language and C#), and
published by Microsoft;
New languages, dedicated to make writing JavaScript programs
easier, with an alternative syntax and new concepts (like classes, lambdas,
scoping, splats, comprehensions...): most relevant languages of this family are
CoffeeScript and Dart;
High-level languages, like Google
Web Toolkit (compiling Java code), JSIL (from C# via Mono), or
Smart Mobile Studio (from
object pascal);
Low-level languages, like Emscripten
(compiling C/C++ from LLVM byte-code, using asm.js).
Of course, from our point of view, use of modern object pascal is
of great interest, since it will leverage our own coding skills, and make us
able to share code between client and server sides.
Beyond JavaScript
The so-called Smart
Pascal language brings strong typing, true OOP to JavaScript,
including classes, partial classes, interfaces, inheritance, polymorphism,
virtual and abstract classes and methods, helpers, closures, lambdas,
enumerations and sets, getter/setter expressions, operator overloading,
contract programming. But you can still unleash the power of
JavaScript (some may say "the good parts"), if needed: the
variant type is used to allow dynamic typing, and you can write
some JavaScript code as an asm .. end block.
See http://en.wikipedia.org/wiki/The_Smart_Pascal_programming_language
The resulting HTML5 project is self-sufficient with no external
JavaScript library, and is compiled as a single
index.html file (including its css, if needed). The
JavaScript code generated by the compiler (written in Delphi
by Eric Grange), is of very high quality, optimized for best execution
performance (either in JIT or V8), has low memory consumption, and can be
compressed and/or obfuscated.
The SmartCL runtime library encapsulate HTML5 APIs in a set of
pure pascal classes and functions, and an IDE with an integrated form designer
is available. You can debug your application directly within the IDE (since
revision 2.1 - even if it is not yet always stable) or within your browser (IE,
Chrome or FireBug have great debuggers), with step-by-step execution of the
object pascal code (if you define "Add source map (for debugging)" in
Project Options / Linker).
Using a third-party tool like PhoneGap - see http://phonegap.com - you would be able to supply
your customers with true native iOS or Android applications,
running without any network, and accessing the full power of any modern
Smart Phone. Resulting applications will be much smaller in size than
the one generated by Delphi FMX (a simple Smart RESTful
client with a login form and ORM + SOA tests is zipped as 40 KB), and will work
seamlessly on all HTML5 platforms, including most mobile (like Windows Phone,
Blackberry, Firefox OS, or webOS) or desktop (Windows, Linux, BSD, MacOS)
architectures.
Smart Mobile Studio is therefore a great platform for implementing
rich client-side AJAX or Mobile applications, to work with our
client-server mORMot framework.
Using Smart Mobile Studio with mORMot
There is no package to be installed within the Smart Mobile Studio
IDE. The client units will be generated directly from the mORMot
server.
Any edition of Smart - see http://smartmobilestudio.com/feature-matrix
- is enough: you do not need to pay for the Enterprise edition to
consume mORMot services. But of course, the Professionnal
edition is recommended, since the Basic edition does not allow to
create forms from the IDE, which is the main point for an AJAX application.
In contrast to the wrappers available in the
Entreprise edition of Smart, for accessing RemObjects or
DataSnap servers, our mORMot clients are 100% written in the
SmartPascal dialect. There is no need to link an external
.js library to your executable, and you will benefit of the
obfuscation and smart linking features of the Smart compiler.
The only requirement is to copy the mORMot cross-platform units to
your Smart Mobile Studio installation. This can be done in three
copy instructions:
xcopy SynCrossPlatformSpecific.pas "c:\ProgramData\Optimale Systemer AS\Smart Mobile Studio\Libraries" /Y
xcopy SynCrossPlatformCrypto.pas "c:\ProgramData\Optimale Systemer AS\Smart Mobile Studio\Libraries" /Y
xcopy SynCrossPlatformREST.pas "c:\ProgramData\Optimale Systemer AS\Smart Mobile Studio\Libraries" /Y
You can find a corresponding BATCH file in the CrossPlatform
folder, and in SQLite3- SmartMobileStudio
ClientCopySynCrossPlatformUnits.bat.
In fact, the SynCrossPlatformJSON.pas unit is not used under
Smart Mobile Studio: we use the built-in JSON serialization features
of JavaScript, using variant dynamic type, and the
standard JSON.Stringify() and JSON.Parse()
functions. [Less]
|
Posted
about 11 years
ago
by
A.Bouchez
By default, interface-based services of a mORMot server will always
return a JSON array (or a JSON object, if
TServiceFactoryServer.ResultAsJSONObject is
true).
With some kind of clients (e.g. if they are made by a third party), it could be
useful to
... [More]
return XML content instead.
Your mORMot server is able to let its interface-based
services return XML context instead, or in addition to the default JSON
format. Always return XML content
If you want all methods of a given interface to return XML
content instead of JSON, you can set
TServiceFactoryServer.ResultAsXMLObject to true.
Instead of this JSON array content, returned by default:
GET root/Calculator/Add?n1=1&n2=2
...
{"result":[3]}
or this JSON object, if ServiceFactoryServer.ResultAsJSONObject
is true:
GET root/Calculator/Add?n1=1&n2=2
...
{"result":{"Result":3}}
The following XML will be returned if
TServiceFactoryServer.ResultAsXMLObject is true:
GET root/Calculator/Add?n1=1&n2=2
...
<?xml version="1.0" encoding="UTF-8"?>
<result><Result>3</Result></result>
Conversion is processed from the JSON content generated by the
mORMot kernel, via a call to JSONBufferToXML() function,
which performs the XML generation without almost no memory allocation. So only
a slightly performance penalty may be noticed (much faster in practice than
most node-based XML producers available).
One drawback of using this
TServiceFactoryServer.ResultAsXMLObject property is that your
regular Delphi or AJAX clients won't be able to consume the service any more,
since they expect JSON content.
If you want your service to be consumed either by XML and JSON, you would need
two services. You can therefore define a dedicated interface to
return XML, and then register this interface to return only XML:
type
ICalculator = interface(IInvokable)
['{9A60C8ED-CEB2-4E09-87D4-4A16F496E5FE}']
/// add two signed 32 bit integers
function Add(n1,n2: integer): integer;
end;
ICalculatorXML = interface(ICalculator)
['{0D682D65-CE0F-441B-B4EC-2AC75E357EFE}']
end; // no additional method, just new name and GUID
TServiceCalculator = class(TInterfacedObject, ICalculator,ICalculatorXML)
public // implementation class should implement both interfaces
function Add(n1,n2: integer): integer;
end;
...
aServer.ServiceRegister(TServiceCalculator,[TypeInfo(ICalculator)],sicShared);
aServer.ServiceRegister(TServiceCalculator,[TypeInfo(ICalculatorXML)],sicShared).
ResultAsXMLObject := True;
...
There would therefore be two running service instances (e.g. here two
instances of TServiceCalculator, one for ICalculator
and one for ICalculatorXML). It could be an issue, in some
cases.
And such a dedicated interface may need more testing and code on the server
side, since they will be accessible from two URIs:
GET root/Calculator/Add?n1=1&n2=2
...
{"result":{"Result":3}}
and for ICalculatorXML interface:
GET root/CalculatorXML/Add?n1=1&n2=2
...
<?xml version="1.0" encoding="UTF-8"?>
<result><Result>3</Result></result>
Return XML content on demand
As an alternative, you can let the mORMot server inspect the
incoming HTTP headers, and return the content as XML if the
"Accept: " header is exactly "application/xml" or
"text/xml".
You can set the
TServiceFactoryServer.ResultAsXMLObjectIfAcceptOnlyXML property to
enable this HTTP header detection:
aServer.ServiceRegister(TServiceCalculator,[TypeInfo(ICalculator)],sicShared).
ResultAsXMLObjectIfAcceptOnlyXML := true;
For standard requests, the incoming HTTP header will be either void, either
"Accept: */*", so will return JSON content.
But if the client set either "Accept: application/xml" or
"Accept: text/xml" in its header, then it will return an XML
document.
Instead of this JSON content:
GET root/Calculator/Add?n1=1&n2=2
Accept: */*
...
{"result":{"Result":3}}
The following XML will be returned:
GET root/Calculator/Add?n1=1&n2=2
Accept: application/xml
...
<?xml version="1.0" encoding="UTF-8"?>
<result><Result>3</Result></result>
as it would with "text/xml":
GET root/Calculator/Add?n1=1&n2=2
Accept: text/xml
...
<?xml version="1.0" encoding="UTF-8"?>
<result><Result>3</Result></result>
Note that the header is expected to be "Accept:
application/xml" or "Accept: text/xml" as exact
value.
For instance "Accept: text/html,application/xml,*/*" won't be
detected by the server, and will return regular JSON:
GET root/Calculator/Add?n1=1&n2=2
Accept: text/html,application/xml,*/*
...
{"result":{"Result":3}}
Your XML client should therefore be able to force the exact content of the
HTTP "Accept:" header.
Together with parameter values optionally encoded at URI level - available
with TSQLRestRoutingREST default routing scheme (see
?n1=1&n2=2 above)- it could be an useful alternative to
consume mORMot services from any XML-based client.
Feedback is welcome on our forum, as
usual! [Less]
|