I did a simple page display which get back the backtrace from a failed job (adds also a field into datasync_jobs table to store the backtrace if there is).
It displays the link to this page in the watchdog entry which says that the job failed.
| Comment | File | Size | Author |
|---|---|---|---|
| #15 | datasync_backtrace-200910161015.tar_.gz | 5.68 KB | pounard |
| #13 | datasync_backtrace-200910150123.tar_.gz | 8.38 KB | pounard |
| #10 | datasync-backtrace-545934.patch | 6.66 KB | andrewlevine |
| #8 | datasync_backtrace-dev-200910071221.tar_.gz | 1.92 KB | pounard |
| datasync-6.x-1.0-alpha3-patched-backtrace.patch | 4.05 KB | pounard |
Comments
Comment #1
pounardIt was patched over a version with #536528: DataSync should catch exceptions and mark the job as failed and #536512: Datasync should not set the memory_limit itself patches applied, but I don't think it should break if you don't have the last one.
Comment #2
andrewlevine commentedThis is a great idea and something I was thinking about from the beginning. There should definitely be a way to monitor failures (and get notified about them in different ways). However, I'm not sure I agree with the approach. Here's what I think:
I definitely want this functionality to get in. Thanks again for all your contributions Pierre
Comment #3
pounard1) I agree
2)
I don't think so. First implementation on my side was on a separate table. The fact it was unnecessary for only field.3) Ok, with an implementation with a hook your point of view on 2) is better.
The thing is, you have to do a delete operation on hook, when you delete a job, don't keep trace of backtrace, so, this need a datasync_job_delete() function implentation (like I did in drush integration file).
Comment #4
andrewlevine commentedAbout deletion, the $job->remove() method calls hook_datasync_job_removed(), so any deletion can be done there
Comment #5
pounardOk, thanks
Comment #6
pounardHello again!
I'm actually rewriting this feature as a new module. I wish you could review it when I'll upload it here. I wonder if you could create a contrib/ sub dir in your module repository and allow me to put this as a submodule, and the drush integration (#541784: Drush integration) with it?
The fact I'm not fond of creating a new module project for each of these single features.
What do you think?
Comment #7
pounardNote about datasync.job.inc
Lines 141 to 147:
This code should also export $e backtrace so I could get it in the hook_datasync_exception_thrown(). The main goal of this submodule is not debug DataSync itself, but debug the custom implementation.
Comment #8
pounardPlease review and try this.
Comment #9
pounardComment #10
andrewlevine commentedPierre,
This is great. I've made a few changes - some changes to the copy, some drupal coding standards changes, and a couple bugfixes. The patch to DataSync is attached below.
I think it would be a killer feature if we extended this module beyond exception catching. Maybe we change the name to datasync_log and we implement many of the other hooks (hook_datasync_phase_failed, datasync_job_removed, datasync_job_added, datasync_phase_succeeded, datasync_phase_postponed, etc.). The user should be able to configure which events are logged. Because the events are logged with watchdog, they can have the information of whatever job events they want emailed to them, written to a logfile, etc. We then add a view like the /admin/settings/datasync/jobs page to show all the log messages.
What do you think?
Also, I see what you mean about the hook_datasync_exception_thrown hook being confusing and am going to create an issue for it
Comment #11
pounardI totally agree with the configuration granularity you are proposing. There is a lot to do with this submodule. I'll take some time as soon as I can to continue improvements on this module.
Comment #12
pounardWhen you are doing this:
You are catching every exception that may happen in the custom job implementation. In my case, I throw some custom exceptions (EntityException in my module) that I want to be catched by your module if I forgot the right catch statement in my job implementation (plus, some PHP low level exceptions could be thrown too).
The thing that is wrong in this code is you get back exception message but not the backtrace of the custom exception (remember that my exception is not a DataSync exception, but a custom exception which comes from an API which is not DataSync related).
I don't have my working laptop with me right now, so I can't do a quick and dirty patch to show you, but I hope this clarify what I try to explain.
What I want to get back here, is the custom exception backtrace, not just the error message, so when you throw the hook_datasync_exception_thrown() I can get it back in the submodule to log it.
Remember that I does this module to debug my custom jobs, not to debug DataSync itself. But the idea to do a granular multi-level logging is still a really good idea.
Comment #13
pounardPlease try this one, and tell what do you think.
EDIT: oups, I'm tired, rename the file to .tar.gz (typo error)
Re EDIT: please avoid diff if you modify it, I did put the module into a different folder this is like hell to apply:)
Comment #14
pounardThis version is quite buggy, I'll send a new one soon (I'm currently workig with, I really needed this feature).
Comment #15
pounardHere is an updated version, with no known bugs for me.
Comment #16
pounard@andrewlevine
Did you looked at the code update?
Comment #17
andrewlevine commentedI'm still trying to get to this. I've looked through it a few times and it's 90% there, I just want to make sure I'm comfortable with the code before I give you my patch based on yours and eventually commit it.
Comment #18
pounardOk, there is no hurry here, I'm currently using the code I gave you on some sites. But whenever you create a new datasync release I just want to make sure that day my code will continue to work :)
It's not flawless, and I think it can be better, I need you to tell where I do a wrong usage of your own hooks and if you're comfortable with the way I do all those things.