Lustre & SQLite

[Michal Wesolowski]

Hi Experts

I'm trying to determine whether SQLite will properely work on Lustre. From a
quick search it seems that some people use it successfully using
flock/localflock mount option. But I have doubts.
I'm interested in 2 scenarios:
- SQLite works properely for single Lustre node.
- SQLite files are accessed simultanously by processes on more than one node.

In SQLite documentation (http://www.sqlite.org/lockingv3.html) there is
statement that this db engine uses advisory locks to protect shared date. I know
Lustre doesn't implement this yet. So my questions are:
- Am I right suspecting that for current Lustre version (1.6.6) there is real
risk of SQLite files corruption (in a sense of their integrity - not from fs
point of view) even if using flock option?
- Are advisory locks still planned to come with 1.8 version (as it is indicated
in current Lustre Operation Manual, but there is no such feature in November
Roadmap slide on wiki.lustre.org site)?

Thanks in advance.

Best Regards

Michal Wesolowski

[Oleg Drokin]

Hello!

Post by Michal Wesolowski
In SQLite documentation (http://www.sqlite.org/lockingv3.html) there is
statement that this db engine uses advisory locks to protect shared date. I know
- Am I right suspecting that for current Lustre version (1.6.6) there is real
risk of SQLite files corruption (in a sense of their integrity - not from fs
point of view) even if using flock option?
- Are advisory locks still planned to come with 1.8 version (as it is indicated
in current Lustre Operation Manual, but there is no such feature in November
Roadmap slide on wiki.lustre.org site)?

Lustre supports posix advisory locks. You can enable it with -o flock
mount option
(on all clients).
-o localflock is posix advisory locks with no cluster coherency, so it
is unsafe
for the same database used across several nodes.

Bye,
Oleg

[Michal Wesolowski]

Post by Oleg Drokin
Hello!

Post by Michal Wesolowski
In SQLite documentation (http://www.sqlite.org/lockingv3.html) there is
statement that this db engine uses advisory locks to protect shared date. I know
- Am I right suspecting that for current Lustre version (1.6.6) there is real
risk of SQLite files corruption (in a sense of their integrity - not from fs
point of view) even if using flock option?
- Are advisory locks still planned to come with 1.8 version (as it is indicated
in current Lustre Operation Manual, but there is no such feature in November
Roadmap slide on wiki.lustre.org site)?

Lustre supports posix advisory locks. You can enable it with -o flock
mount option
(on all clients).
-o localflock is posix advisory locks with no cluster coherency, so it
is unsafe
for the same database used across several nodes.
Bye,
Oleg

Thanks Oleg

My interpretation of Lustre documentation was that flock/localflock implements
only mandatory locks.
So I assume statement in Lustre Manula (ver. 15, page 16-1):

Note – Advisory fcntl/flock/lockf locks will be available in Lustre 1.8.

is a documentation bug, isn't it?

Thanks

Michal

[Oleg Drokin]

Hello!

Post by Michal Wesolowski
My interpretation of Lustre documentation was that flock/localflock
implements only mandatory locks.

There is no such thing as posix mandatory locking, I think.
The existing mandatory locking depends on a combination of advisory
locking and kernel support in
interpretation of a special file mode set.

Post by Michal Wesolowski
Note – Advisory fcntl/flock/lockf locks will be available in Lustre 1.8.
is a documentation bug, isn't it?

Yes, this is somewhat outdated.
1.6.6 and above have posix advisory locking working (though
performance is not ideal,
especially if you have many clients doing a lot of locks on the same
file at the same
time).

Bye,
Oleg

[Michal Wesolowski]

Thanks Oleg !

I have subquestion regarding locking performance, please see below.

Post by Oleg Drokin
Hello!

Post by Michal Wesolowski
My interpretation of Lustre documentation was that flock/localflock
implements only mandatory locks.

There is no such thing as posix mandatory locking, I think.
The existing mandatory locking depends on a combination of advisory
locking and kernel support in
interpretation of a special file mode set.

Post by Michal Wesolowski
Note – Advisory fcntl/flock/lockf locks will be available in Lustre 1.8.
is a documentation bug, isn't it?

Yes, this is somewhat outdated.
1.6.6 and above have posix advisory locking working (though performance
is not ideal,
especially if you have many clients doing a lot of locks on the same
file at the same
time).

Do you mean file locking mechanism in Lustre doesn't perform as efficiently as
on local system or is not yet optimized? Or is there any general impact on
Lustre when at least one node has flock option turned on? For example additional
burden on MDS or OSSes which results in lower IO/s performance.

Regards

Michal

[Oleg Drokin]

Hello!

Post by Michal Wesolowski

Post by Oleg Drokin
Yes, this is somewhat outdated.
1.6.6 and above have posix advisory locking working (though
performance is not ideal,
especially if you have many clients doing a lot of locks on the
same file at the same
time).

Do you mean file locking mechanism in Lustre doesn't perform as
efficiently as on local system or is not yet optimized? Or is there
any general impact on Lustre when at least one node has flock option
turned on? For example additional burden on MDS or OSSes which
results in lower IO/s performance.

It is both not very optimized and slower than local system since it
needs to send network rpcs for locking (Except for the localflock
which is same speed as for local fs).
The performance impact would only be realized if not only you mount
with -o flock, but actually use posix locking, and mostly only in
those operations.
There is no impact on OSS operations at all. The only MDS impact is
there is now a bit more processing to handle those locking RPCs.

Bye,
Oleg

[Michal Wesolowski]

Oleg, thanks for your support

m.

Post by Oleg Drokin
Hello!

Post by Michal Wesolowski

Post by Oleg Drokin
Yes, this is somewhat outdated.
1.6.6 and above have posix advisory locking working (though
performance is not ideal,
especially if you have many clients doing a lot of locks on the
same file at the same
time).

Do you mean file locking mechanism in Lustre doesn't perform as
efficiently as on local system or is not yet optimized? Or is there
any general impact on Lustre when at least one node has flock option
turned on? For example additional burden on MDS or OSSes which
results in lower IO/s performance.

It is both not very optimized and slower than local system since it
needs to send network rpcs for locking (Except for the localflock
which is same speed as for local fs).
The performance impact would only be realized if not only you mount
with -o flock, but actually use posix locking, and mostly only in
those operations.
There is no impact on OSS operations at all. The only MDS impact is
there is now a bit more processing to handle those locking RPCs.
Bye,
Oleg
_______________________________________________
Lustre-discuss mailing list
http://lists.lustre.org/mailman/listinfo/lustre-discuss

[Michal Wesolowski]

Post by Oleg Drokin
Hello!

Post by Michal Wesolowski
In SQLite documentation (http://www.sqlite.org/lockingv3.html) there is
statement that this db engine uses advisory locks to protect shared date. I know
- Am I right suspecting that for current Lustre version (1.6.6) there is real
risk of SQLite files corruption (in a sense of their integrity - not from fs
point of view) even if using flock option?
- Are advisory locks still planned to come with 1.8 version (as it is indicated
in current Lustre Operation Manual, but there is no such feature in November
Roadmap slide on wiki.lustre.org site)?

Lustre supports posix advisory locks. You can enable it with -o flock
mount option
(on all clients).
-o localflock is posix advisory locks with no cluster coherency, so it
is unsafe
for the same database used across several nodes.
Bye,
Oleg

Thanks Oleg

My interpretation of Lustre documentation was that flock/localflock implements
only mandatory locks.
So I assume statement in Lustre Manula (ver. 15, page 16-1):

Note – Advisory fcntl/flock/lockf locks will be available in Lustre 1.8.

is a documentation bug, isn't it?

Thanks

Michal

[Andreas Dilger]

Post by Michal Wesolowski
I'm trying to determine whether SQLite will properely work on Lustre. From a
quick search it seems that some people use it successfully using
flock/localflock mount option. But I have doubts.
- SQLite works properely for single Lustre node.
- SQLite files are accessed simultanously by processes on more than one node.

I think that is only between processes on the same node. I think accessing
SQLite between different processes on different nodes might result in file
corruption by SQLite itself, regardless of the locking by Lustre.

If you want to test this, you definitely need to mount all clients with
"-o flock" so the locking is coherent across all clients.

Post by Michal Wesolowski
In SQLite documentation (http://www.sqlite.org/lockingv3.html) there is
statement that this db engine uses advisory locks to protect shared date.

6.0 How To Corrupt Your Database Files

SQLite uses POSIX advisory locks to implement locking on Unix...
Your best defense is to not use SQLite for files on a network filesystem.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

[Brian J. Murrell]

Post by Andreas Dilger
I think that is only between processes on the same node. I think accessing
SQLite between different processes on different nodes might result in file
corruption by SQLite itself, regardless of the locking by Lustre.

Can you expand on why you think this might be the case Andreas? If the
locking works, and the application obeys the locks, what avenues do you
see for corruption?

Post by Andreas Dilger
6.0 How To Corrupt Your Database Files
SQLite uses POSIX advisory locks to implement locking on Unix...
Your best defense is to not use SQLite for files on a network filesystem.

Andreas, do you think there is any scientific data behind this kind of
statement or do you think it's just FUD given NFS' long history of
broken file locking?

b.

[Michal Wesolowski]

Post by Andreas Dilger

Post by Michal Wesolowski
I'm trying to determine whether SQLite will properely work on Lustre. From a
quick search it seems that some people use it successfully using
flock/localflock mount option. But I have doubts.
- SQLite works properely for single Lustre node.
- SQLite files are accessed simultanously by processes on more than one node.

I think that is only between processes on the same node. I think accessing
SQLite between different processes on different nodes might result in file
corruption by SQLite itself, regardless of the locking by Lustre.
If you want to test this, you definitely need to mount all clients with
"-o flock" so the locking is coherent across all clients.

Post by Michal Wesolowski
In SQLite documentation (http://www.sqlite.org/lockingv3.html) there is
statement that this db engine uses advisory locks to protect shared date.

6.0 How To Corrupt Your Database Files
SQLite uses POSIX advisory locks to implement locking on Unix...
Your best defense is to not use SQLite for files on a network filesystem.

I saw this too, but my impression was that this remark apply to previous
deliberations about buggy implementation of advisory locks on some NFS systems.

regards

michal