'wrong' event being caught for file creation in File::ChangeNotify::Watcher::Inotify

[autarch]

Migrated from rt.cpan.org #109660 (status was 'open')

Requestors:

• [email protected]

From [email protected] on 2015-11-25 13:35:49
:

Hi there Mr Rolsky,

I have been using the File::ChangeNotify module in some recent work for
my employer.

I have tried using both File::ChangeNotify::Watcher::Default and
File::ChangeNotify::Watcher::Inotify. This problem affects both in a
sense, but is really an issue with File::ChangeNotify::Watcher::Inotify.

Some of the files I was working with were on the bulky side but not
humunguous - ca. 220MB. But I was finding that when I wrote code to
check the file size after the watcher had detected it, these files were
failing a validity tests that they should be over 66 lines. We came to
realise that the watcher was firing the code before its contents had
finished being copied.

Our lead dev figured out that the event you are picking up in
::Watcher::Inotify via Linux::Inotify2, IN_CREATE, was the issue. This
event gets fired at the beginning of file creation. Hence, before the
contents start being written.

I tried going back to just using ::Watcher::Default, but the same issues
were occurring there, too. Of course, this is unrelated to the inotify
event, as inotify is not involved.

In order to get the desired functionality, I simply replaced all
instances of 'IN_CREATE' in the ::Watcher::Inotify code with
'IN_CLOSE_WRITE'. (I also made some POD changes and renamed the changed
module). Now that I am watching for this event, I don't seem to be
having any more problems with large files appearing to have zero lines.

This should also address some of the issues mentioned in ticket #52325
regarding being able to run code at the end of a file transfer process.

However, IN_CLOSE_WRITE also gets triggered when an existing file is
overwritten, but not when renamed or moved. For my purposes, this was
appropriate. But it does stretch the concept of file creation a little.

Would you consider simply changing ::Watcher::Inotify to watch for this
event in place of IN_CREATE?

Thanks & regards,
Ludovic Tolhurst-Cleaver

Perl Engineer,
Inview Technology Ltd.,
Northwich, England.

[autarch]

From [email protected] (@autarch) on 2015-11-27 06:51:41
:

On Wed Nov 25 08:35:49 2015, [email protected] wrote:

Would you consider simply changing ::Watcher::Inotify to watch for this
event in place of IN_CREATE?

I don't think that'd be quite right. Reading the inotify man page, it looks like IN_CLOSE_WRITE will fire on both create & modify events. I think what I need to do is catch create & modify and then wait for a corresponding IN_CLOSE_WRITE against the same file before passing the event on.

[autarch]

From [email protected] on 2015-11-30 10:15:47
:

I agree - slightly more work, but that would of course allow you to
accurately check that a new file has been created and fully written.

Thanks for getting back to me.

(BTW, thanks to you and the DateTime contributors for writing and
maintaining DateTime - probably my favourite CPAN module!)

Ludo

On 27/11/15 06:51, Dave Rolsky via RT wrote:

<URL: https://rt.cpan.org/Ticket/Display.html?id=109660 >

On Wed Nov 25 08:35:49 2015, [email protected] wrote:

Would you consider simply changing ::Watcher::Inotify to watch for this
event in place of IN_CREATE?
I don't think that'd be quite right. Reading the inotify man page, it looks like IN_CLOSE_WRITE will fire on both create & modify events. I think what I need to do is catch create & modify and then wait for a corresponding IN_CLOSE_WRITE against the same file before passing the event on.