Hi all,
I needed "deep" integration between Drupal and Knowledgetree. I've built a KT integration library that interfaces with KT via the new REST API (available in KT3.6). For now it only has 1 or 2 functions but it does all the REST calls and all the session stuff so you can do very simple KT integration like:
$kt = new KT_Session;
$kt->login();
$kt->get_document_details("1"); //returns an array which is the document, document_id=1
$kt->logout();
With this simple REST library I have implemented a CCK integration field. It offers a new CCK field type called "KnowledgeTree Document Reference" which works a bit like "Node Reference". This means you can add this field to a node and then add one or more KT document references. When the node is displayed it can offer a link to view the details (takes you to the KT page for that document) or to download (download via KT).
The basic concept can easily be extended to use many of the REST API features of KT, such as showing a document's workflow (from KT) in a drupal View.
This is a completely different approach to KT integration from this module. It doesn't replace this "KT inside Drupal" approach and it only accesses KT as a single user (so whatever that user can access in KT can be accessed via Drupal). The advantage of what I have built is that it allows KT documents to appear inside Drupal nodes as if they are "native". The REST API also does all the heavy lifting making it very very easy to extend it to other functions that KT offers (folders, actions, documents, workflow etc).
Here are my questions:
Should I look to make my REST API and CCK "KT document reference" functions part of this module by contributing code here?
Or should I fork a different project for this purpose?
Does KT CCK integration interest anyone else?
Does KT REST based API integration (only in KT 3.6) interest anyone else?
If I get some positive feedback I will share some code. It is tested and working (it's somewhat crude). I want some feedback before I dump code into the issues page of this project, because I don't want to distract, confuse or fork off without some feedback from the developers of this module.
| Comment | File | Size | Author |
|---|---|---|---|
| #7 | kt-1.3-kt_api-r59.diff | 111.49 KB | aantonop |
| #5 | kt_new_cck_and_rest.tar_.gz | 26.29 KB | alexpott |
| #4 | kt_REST-6.x-dev-0.38.tgz | 13.85 KB | aantonop |
| #3 | kt_REST-6.x-dev-0.33.tar_.gz | 10.82 KB | aantonop |
Comments
Comment #1
alexpottSounds great to me. In fact I started working on something similar a while ago but never got the time to get a development release ready. Rather than creating a fork I would favour you becoming a co-maintainer and this functionality either being coded into the existing module or going into it's own module as part of the KT integration suite. I'm more than happy to have a look at your code too.
It's great to see people interested in this module!
Alex
Comment #2
aantonop commentedok, sounds good
It's pretty much standalone now, so I will clean it up and submit a patch that will create it as 6-7 new files in the modules directory.
thanks
Comment #3
aantonop commentedHere's some tested and working code. It installs as a standalone module called kt_REST. It provides a test page in the main menu that retrieves the document with ID "1". It also provides a docreference CCK field. Very basic module, should not be installed on a production site, has no security! Install only on a test or dev site to play with the REST API and the CCK field.
- KT Integration: Change the URL, username and password in kt_REST.module
- debug: Uses devel module's dpm function. Change it or comment out in kt_REST.module
- REST and SOAP libraries. The SOAP library is entirely auto-generated and untested. The REST library is tweaked, tested and working smoothly. Both are from the WSDL form KT 3.6.1. REST only works KT >= 3.6
- Testing: Open the KT Test Page in the menu. Should show information about document with ID "1" from your KT installation.
- Docreference: Create a new content type, add a CCK docreference field, create a node, put "1" in the docereference, save, view. Play with CCK's display fields to change how it is displayed.
Give me lots of feedback
This is in very active development. I'm now working on an autocomplete widget to enter KT documents in docreference via a search.
Comment #4
aantonop commentedSignificantly better and easier to test code:
Still a standalone module "kt_REST", no dependencies on kt module.
Changes:
- Settings page with KT URL, username, password, cache timeout and debug option
- Page for testing settings is moved to admin/settings/kt_REST/test under settings page.
- ** Autocomplete ** field for docreference, using search API call to search for documents containing the autocomplete string in the Title or Filename. Much easier to attach KT documents to nodes now. Works with multiple (including unlimited) instances.
- Fixed logging method to use drupal_set_message so no more dependency on devel module
- Fixed session caching issues
It's gone through quite a lot of testing. I am about to start using this on a prototype for a production site, as of Monday.
Next: Test and enhance security. Start discussion on how to leverage this code or integrate into kt module.
Feedback and comments welcome, testing very welcome! Please do not put this on a production site unless you really understand the code and security implications, it has pretty light security right now.
Comment #5
alexpottHi,
Just having a look at this - I want to add all the work together in a sensible way. So I propose:
The kt_REST.module becomes kt_api.module and provides exactly the same functionality that you've coded.
The original iframe providing module stays as kt.module (so upgrade is easy) but depends on the new kt_api.module so that the root path only has to be configured in one place.
I've attached the proposed changes - if you agree I'll commit the lot to the latest dev version.
I think one limitation of the current cck implementation might be that when you download a document using it knowledge tree will log the download as being by the user configured on the api settings page rather than the drupal user.
I really like the cck field... excellent idea.
Cheers,
Alex
Comment #6
aantonop commentedAlex,
I'll take a look at your code and make the necessary changes. I'm using SVN on my end for code revision control. I can then submit a patch against the main kt code from CVS (HEAD?) for you to test?
My TODO for kt_docreference.module
- Throaway download link for specific roles - use the REST API to generate a single-use, auto-expiring URL for downloading a document without KT credentials. This can be used to expose KT CCK attachments to users without credentials
- Reverse document links, so that when you attach a KT doc in drupal it adds external links to the document in KT pointing back to the node in drupal. That way you can easily jump back and forth
- Make these "reverse" links a checkbox option in the CCK field configuration so you can enable/disable linking
My TODO for both kt_docreference and kt_api (as I will now call it):
- Make logins user specific. Rather than depending on a plugin in KT, I was thinking of either storing KT credentials with the user in drupal, or enabling LDAP integration. Either way, I will make each call to "login" check if the user has a KT account. If they do, then it will log in as them, if they don't it will log in with the drupal account.
- Add multiple session authentication options, per drupal role. For example:
role "Staff" => per user session logins (either LDAP or stored credential),
role "Contractors" => shared KT "contractor" login,
role "authenticated" =>shared KT "authenticated" login
role "anonymous" => KT anonymous login.
That way you can map any role in drupal to an anonymous login, a predefined shared KT account (define as many as you like), or a per-user credential for users of that role.
What I would like to do is see how we can merge this authentication model with your frame system. So that the frame logs in using the REST login method, which returns a session ID for you, and based on the above mappings of role->authentication method.
My end goal would be to do all the authentication in drupal making the KT authenticator plugin unnecessary. The reason for this is that it is the hardest part of the KT integration installation process and if we can avoid it then this module will really take off. If we can make the changes above, I think we can integrate drupal and KT with absolutely NO changes in KT, and only a simple self-installing module in drupal
What do you think?
Comment #7
aantonop commentedI re-did the renaming and tested it, plus added all my latest changes and fixes. This is release 0.59 of kt_REST, now renamed kt_api and kt_docreference.
The diff puts them under the original kt module. It creates three new subdirectories for the kt module:
- kt/includes - the include files with the REST API, SOAP API and class definitions
- kt/kt_api - the drupal module which provides the REST API functions
- kt/kt_docreference - a KT CCK field using kt_api
It's a bit cleaner with each in their own subdirectories.
The KT CCK field code is going into production at my company next week. I will continue developing features and expect to find some more bugs. So far however, it has undergone a lot of testing and appears to work quite well.
Comment #8
adideswar commentedGreetings..
Wow, i think your working module meet the urgent requirements in my development site. But you see instead a developer, i am just a designer (using worked drupal module). Can you give me some explanation how to use your module in my site? because usually drupal module is ini tar/gz files and i just dropped in sites/all/modules. Or maybe you can give me some explanation how to implement this code in my site. I understand PHP and SQL a little.
Will very appreciate your time even a little. :) Thanks.
Regards,
adi
Comment #9
alexpott#8 this is experimental code - that will go into the development release very soon...
It is a diff which will create the new modules when applied against the kt directory. Read http://drupal.org/patch/apply to find out how to apply.
Comment #10
aantonop commentedThe code is now in production in our company. It is being tested (bombarded) daily by 14 people. No bugs or snafus so far.
one thing to note is that I organized it under it's own "project" in the install files, instead of "other". Perhaps you can change the kt install file so that it all groups together? the KT docreference part of course appears under the CCK category.
alex, what else do you need from me to get this into the dev release? Any help needed?
I used ot have CVS creds a long time ago but I think they've probably lapsed. Perhaps I should get new ones to help maintain this. We would probably need to update the project page too to document the new modules and capabilities. Sooner we make available and document the sooner people can try testing it.
Andreas
Comment #11
adideswar commentedHi guys, thanks very much for your time explaining to me.
I have tried one of your module in #5 (http://drupal.org/files/issues/kt_new_cck_and_rest.tar_.gz) and i have test the kt_rest by going to this link (http://localhost/drupal/?q=admin/settings/kt/REST_test), which written:
"If this test is successful you should see an array with document details matching the document with ID#1 in KnowledgeTree: "
but in my site it wasnt succesful, above it's written:
"warning: strtr() [function.strtr]: The second argument is not an array in C: \ms4w\Apache\htdocs\drupal\modules\syslog\syslog.module on line 106."
i've used kt about two weeks and many activities for trying KnowledgeTree since that time. The document with id#1 it's not exist anymore because i've deleted it.
What could be problem of my site? Should i reinstall fresh KT to make your kt_rest module works?
i'm sorry since i think i'm the most stupid person in this issues. :(
Comment #12
alexpottJust committed the changes. Should be in dev sometime soon.
Changes to aantonop's code and the original module are:
Certainly not against aantonop becoming a co-maintainer for the module.
Comment #13
aantonop commentedRE: adideswar #11
A tester, OMG, an actual tester!
Don't give up yet, we will help!
ok, seriously though. I'm gonna make some changes to the test page so that it doesn't just pull doc #1, since in many cases it will not exist. Let me think of a better test.
In the mean time, have you got the debug setting turned on? That's really where the action happens. If you turn on the debug it will dump a whole lot of info on your screen when you visit the test page and that will make it easier to see what is happening. (if you do run a test with debug and copy the information here, make sure you obscure your login ID and password, it's in the REST URLs that appear in the debug info!!!)
Secondly, have you tried adding a CCK field to a new node type? Go to Content Types, and either create a new content type or edit an existing one. Go to Manage Fields and add a new field. Call it whatever you want. Pick "KnowledgeTree Document Reference" in the field type and "Autocomplete" for the widget.
Save it and then create a new node of that type. Scroll down till you see the autocomplete field. Start typing something that appears in the filename or title of a document in KT. If a list appears then it's WORKING! yay!
An easy way to test it is to type the letter "e" and wait for a second while the auto-complete twirls and searches for you. "e" is the most common letter in english so the chance of a file or title containing it is pretty high. If you type "e" and nothing comes up, then it's not working. At that point we need to do more troubleshooting.
Let us know how it goes!
Some reminders for me to fix things:
@todo - add more information in the test page to remind people to turn on debug
@todo - create a more general test, like get a session ID, then get the root folder. Avoid doc#1 in case it has been deleted.
@todo - obscure password and login info in debug messages
@todo - create a tutorial for KT cck field
Comment #14
adideswar commented@aantonop
haha,, thanks for helping..
Aye sir, wont give up..:))
I would like to be a tester for this module if it helps you developing since i need the function too :).
Okay, first job, i've debugging the site (though i'm not sure i'm doing debug correctly). I've installed devel and firebugfordrupal module and whenever loading a page, there is Node Access Summary below, i cant copy paste on text so i took snapshot on them for you. The summary when i open page http://localhost/drupal/?q=admin/settings/kt_REST/test is this http://i94.photobucket.com/albums/l87/adideswar/debug.jpg
Tell me if it is the right report you want or not, and if it is not tell me how i can get the right one for you.
As for kt_rest test, eventually its my dumb mistake, i dont add slash in the last for the root of kt, so now the root is http://localhost/dms/. When i run the test again, the report shows the is worked. (All the reports above is yellow in color) And the array below is shown to like this:
Array
(
[document_id] => 1
[custom_document_no] => n/a
[oem_document_no] => n/a
[title] => AdvancedCreateShapefile_id
[document_type] => IT Solution - Midleware - Datawarehouse
[filename] => AdvancedCreateShapefile_id.swf
[filesize] => 1043722
[folder_id] => 0
[created_by] => admin
[created_date] => 2009-07-15 17:23:46
[checked_out_by] => n/a
[checked_out_date] => n/a
[modified_by] => admin
[modified_date] => 2009-07-15 17:23:46
[owned_by] => admin
[version] => 0.1
[content_id] => 1
[is_immutable] =>
[permissions] => RW
[workflow] => n/a
[workflow_state] => n/a
[full_path] => /AdvancedCreateShapefile_id
[mime_type] => application/x-shockwave-flash
[mime_icon_path] => unspecified_type
[mime_display] => SWF File
[storage_path] => 00/1
[metadata] =>
[links] =>
[transitions] =>
[version_history] =>
[transaction_history] =>
)
Second, I've tried adding field as you told. But after i save it, error is shown like this:
* warning: pg_query() [function.pg-query]: Query failed: ERROR: syntax error at or near "(" LINE 1: ...ontent_type_testkt ADD COLUMN field_ktdocfield_value int(11) ^ in C:\ms4w\Apache\htdocs\drupal\includes\database.pgsql.inc on line 139.
* user warning: query: update_sql ALTER TABLE content_type_testkt ADD COLUMN field_ktdocfield_value int(11) in C:\ms4w\Apache\htdocs\drupal\includes\database.pgsql.inc on line 689.
I cant save the field after this, because when i clicked the save field button, the error came like this:
Warning: pg_query() [function.pg-query]: Query failed: ERROR: value too long for type character varying(128) in C:\ms4w\Apache\htdocs\drupal\includes\database.pgsql.inc on line 139
Warning: ERROR: value too long for type character varying(128) query: dblog_watchdog INSERT INTO watchdog (uid, type, message, variables, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', '%message in %file on line %line.', 'a:4:{s:6:"%error";s:7:"warning";s:8:"%message";s:264:"pg_query() [function.pg-query]: Query failed: ERROR: syntax error at or near "(" LINE 1: ...ontent_type_testkt ADD COLUMN field_ktdocfield_value int(11) ^";s:5:"%file";s:56:"C:\\ms4w\\Apache\\htdocs\\drupal\\includes\\database.pgsql.inc";s:5:"%line";i:139;}', 3, '', 'http://localhost/drupal/?q=admin/content/node-type/testkt/fields/field_k...', 'http://localhost/drupal/?q=ad in C:\ms4w\Apache\htdocs\drupal\includes\database.pgsql.inc on line 160
Regardless from error before, i also tried to create content with this field but, when i wanna fill the field with a letter, instead showing document name contain the letter, it shows popup windows like this:
An error occurred.
http://localhost/drupal/?q=kt_docreference/autocomplete/field_ktdocfield
[ ]<script type="text/javascript">jQuery.extend(Drupal.settings, { "thmr_2": { "name": "theme_syslog_format", "type": "func", "duration": 0.14, "used": "theme_syslog_format", "candidates": [ "black_mamba_syslog_format", "phptemplate_syslog_format", "theme_syslog_format" ], "preprocessors": [ ] }, "thmr_1": { "name": "theme_syslog_format", "type": "func", "duration": 3.96, "used": "theme_syslog_format", "candidates": [ "black_mamba_syslog_format", "phptemplate_syslog_format", "theme_syslog_format" ], "preprocessors": [ ] }, "request_id": "11714a852e509c098", "devel_themer_uri": "/drupal/?q=devel_themer/variables/11714a852e509c098" });</script>
Is there any possibilities that the problem is from database? Because i used postgresql for drupal and mysql for kt. Should i use one database for both of kt and drupal?If that so, i will change drupal to mysql too...
Thanks in advance.
Comment #15
alexpott@adideswar
This would appear to be happening because the module doesn't work with postgres... are you using that?
Plus are you using the latest dev version?
Comment #16
alexpottUpdating issues version...
Comment #17
alexpott@adideswar
Try changing the in kt_docreference.module from this
to this
And let us know if that works...
Comment #18
alexpottOops.... actually change it to this
Comment #19
alexpott@adideswar
Once you've changed the code remove the field from your content type and add it back again
Comment #20
aantonop commented@adideswar
Some more clarification:
The "debug" I was referring to is in the settings of the KT REST API module. In there (where you put the URL and user/password) you will see a debug checkbox at the bottom. When that is checked it will output copious debug information on every KT connection. It is very noisy, so only to be used for debugging, not in a production site. Try and see.
If the Test page shows you document #1, as shown above though, you are ALREADY working with KT, so that's awesome. That means the connection and REST part work fine.
Now on the to the field issues:
The module has only been tested with MySQL, which is why it didn't work with Postgress. Alex's changes will probably fix that.
Once you try the database changes alex suggested, try the autocomplete again. If it doesn't work or you can't tell what it's doing, try going directly to the autocomplete page (this is the data the autocomplete Javascript function loads in the background). In your case, it would be this link here:
http://localhost/drupal/?q=kt_docreference/autocomplete/field_ktdocfield
that was seen in your error message. Firebug will show it in the console every time you do an autocomplete. You must add a "search string" to the URL like this
http://localhost/drupal/?q=kt_docreference/autocomplete/field_ktdocfield/e
That will search for the letter "e".
If you try that, see what it returns. If it returns two square brackets "[]" it means it didn't find anything. If it gives you an error, tell us what it is!
Thanks for testing
B
Comment #21
alexpottI've committed the postgres fix to the development release
Comment #22
adideswar commentedSorry for the late reply, just celebrate independence day of my country few days back.
@ alexpott
Yes sir, i used PostgreSQL for Drupal and Mysql for KT. I've just know that the dev version has been released.
Referring to Mr. aantonop's kt_rest module in #4 (which i used until now), fix from Mr. alexpott in #18 was correct. I can put reference field to the kt document now even not changing my database (Postgre for Drupal, and MySQL for KT).
And now, the new problem is the document link. It always refers to "http://localhost/kt/view.php?fDocumentId=[the document id]". The problem is this string: "localhost/kt/". So if the root URL not 'localhost/kt' i can't directly refer to my KT installation. I've changed the Knowledgetree root URL in kt_rest settings to "http://172.27.4.105/dms/" (this is my KT root URL, the IP address sometimes change too) but it doesn't fix this.
@ aantonop
The "debug" already activated, and all messages were in yellow (i assumed that means all worked well ;), CMIIW).
Without change my database, the autocomplete have been run and it worked well (i think :p). I mean when i put letter 'c' in the end of the url like this: "http://172.27.4.105/drupal/?q=kt_docreference/autocomplete/field_ktdocfi...", all document with letter 'c' comes out in the bracket like this:
{ "common [DocumentID:19]": "\x3cdiv class=\"reference-autocomplete\"\x3ecommon\x3c/div\x3e", "nssckbi [DocumentID:18]": "\x3cdiv class=\"reference-autocomplete\"\x3enssckbi\x3c/div\x3e", "PROPOSAL-MaintenanceADD ver2b [DocumentID:9]": "\x3cdiv class=\"reference-autocomplete\"\x3ePROPOSAL-MaintenanceADD ver2b\x3c/div\x3e" }
Dont mention it, it's me who thanks to you all for developing it :)
@ all
since the dev release have come, i will change the testing subject from aantonop's "kt_REST" to "6.x-2.x-dev". is that okay?
Thanks again in advance...
Comment #23
adideswar commentedExcuse me, before changing subject I've check some file in 'kt_REST' folder:
1. Because, the document link didn't refer to my KT installation i looked at the " kt_REST.module " file and found the " $form['kt_root_url'] ". I think the form is not global so i make it global so the document link cant refer to the kt root url.
kt_REST.module
===========
I put the form variables in " function _kt_docreference_details_link " in " kt_docreference_module " file But, i doesn't make a change. Maybe you guys know what's wrong.
2. To get rid of it, i manually created variable to refer to kt root url like this
kt_docreference_module
================
Hahaha, i know this not the correct way to fix, but temporary fulfil the requirement now.
3. A question from me, field knowledgetree doc reference only provide link for view document, is there any field to provide link for directly download the document?
Comment #24
alexpott@adideswar
My fault - I committed an old version of the docreference module. Just committed a new version to fix this. I've also reorganised the module's directory structure so when you install the latest dev remove your current kt folder beforehand. If you don't you'll have multiple versions of the api and docreference hanging around... which would be confusing! After updating you will have to clear Drupal's cache...
In order to get to download the doc you can change the field formatter to present a download link.
Comment #25
adideswar commentedThanks alex,
You said:
"In order to get to download the doc you can change the field formatter to present a download link."
can you tell me step-by-step how to do this? As i know, first I added New field with type Knowledgetree doc reference, then I filled the field with a name of document which (represent the real document with same name), after that the field will ONLY provide A LINK which is view document link. How can i change the link to become download link?
Why it seems only function _kt_docreference_details_link in kt_docreference.module which is implemented in the node?
Can i show some link like, FOR EXAMPLE, if i put a document at the field, after i save the page, the field will show:
* document name (label/no link),
* link for view, and
* link for download?
How can i do that? it seems that in the kt_docreference.module all those function is implemented. How can i used them in the node?
Thanks again.. I'm in your debt..;)
Comment #26
alexpottAs long as you have the latest version of 6.x-2.x-dev you will be able to change how the docreference field in displayed by changing the setting on the "display fields" tab when you are editing a content type (for example admin/content/node-type/page/display)
Comment #27
adideswar commentedthanks, after go to tab "display fields" i must wait for a minute before the configuration shows up (that's why i missed it) :)
Okay, i used the dev version few days back and it works fine (kt rest test and doc reference field works as usual), but today when i go to tab 'kt rest test' it shows up error message like this.
You know what's wrong?
Comment #28
adideswar commentedjust revert to my backup when kt-dev still normal. Because i dont know what's wrong.
Another question:
Why every user in drupal login as only ONE user in KT? (which is set in KnowledgeTree Authentication in admin/settings/kt/api) I know this is from KT REST module by Mr. aantonop , because the earlier version of kt before this is using same username (and also copy their roles) who is login to drupal at same time in KT. I mean for example when we login as 'michael' in drupal, then when we open KT we also login as 'michael' too.
Really no offense, but is it more secure if we login as same user in drupal and KT like the earlier version? Because the earlier version can keep their privileges diffference so it maintain security in dms.
(Bad english, i know, sorry)
Comment #29
aantonop commented@adideswar
You are absolutely right. As it is right now all the kt_docreference logins are as ONE user. I need to implement per-user logins in the next version. It's part of the TODO list (see #6). Unfortunately I have not implemented this. I must review what alex has done with the KT frame to see if I can use some of that to maintain per-user sessions.
No idea when this next step will happen. This development happens in my spare time over weekends, so it's hard to get a lot done.
Thanks for all the testing.
Comment #30
adideswar commented@ aantonop
u'r welcome :)
Just wanna ask little thing, what causing your kt rest test is failed?
it's written:
I think something wrong with the session, but what is it exactly and how i can fix it i have no idea.
At this point I cant login through ktwebservice, i can't use doc reference and i have to revert to my back up again.
when i go to this address:
http://localhost/dms/ktwebservice/KTWebService.php?method=login&username...
it shows like this:
You know why? maybe you can give me little clue..:)
Comment #31
adideswar commentedI know the cause now..
in folder drupal\includes\ file common.inc
this file is changed because i used RSS Feed module (one step of configuration of RSS Feed need to edit this file).
This file must be replaced by an original one.
Thank you very much for Mr. Aantonop and Mr. Alex Pott for developing this KT module.
Comment #32
aantonop commentedHm. That's probably an error with the drupal_http function that is used to make the HTTP request. Something was changed in the way it handled or returned error codes. That then caused the kt REST API to fail.
The XML result shows that a session WAS returned succesfully, so the login worked. But for some reason, the API read the XML return code "0" as an HTTP return code...
Glad you found how to fix it.
Are you using it in production yet?