-
New Commercial Services Section
We have now opened a commercial services section here on the forum for registered users. If you have a Fabrik project that you wish to have someone work on for you, post it under Help Wanted. If you are an application developer and wish to earn some money helping others, post your details under Fabrik Application Developers.
Both of these are unmoderated. It will be up to both parties to work out the details and come to an agreement.
-
Joomla 5.1
For running J!5.1 you must https://fabrikar.com/forums/index.php?wiki/update-from-github/ or include the new file manually https://fabrikar.com/forums/index.php?threads/joomla-5-1-and-fabrik-cannot-find-files-error.54473/post-285151 See also Announcements
[Solution] Import csv results in blank records
- Thread starter Sophist
- Start date
Sophist
Guest
Trying to debug this now and the problem seems to be in components/com_fabrik/models/form.php @ function processToDB.
I am comparing what happens in this function when updating a row with what happens when you import a CSV.
I haven't quite tracked down the problem yet, but I wanted to comment on how this function stores its data. At entry, formData holds a variety of properties, including all placeholders with ___. With the table___element format, each of these is likely to be unique and unlikely to conflict with any other properties stored in formData. However...
It then runs $listModel->removeTableNameFromSaveData to remove the table name and ___ and just leave the field names. This is fine so long as the shortened names are a. unique within the elements being saved (which might not be true) and b. don't conflict with any other properties used by fabrik. But I can easily see how these might NOT be unique. Indeed with common field names such as id it is quite likely for there to be a conflict.
I would recommend that:
1. The removeTableNameFromSaveData returns only placeholders.
2. The returned values are stored separately for each table and separate from other fabrik formData.
I will update this further once I have fully tracked down the problem and submitted a fix PR.
I am comparing what happens in this function when updating a row with what happens when you import a CSV.
I haven't quite tracked down the problem yet, but I wanted to comment on how this function stores its data. At entry, formData holds a variety of properties, including all placeholders with ___. With the table___element format, each of these is likely to be unique and unlikely to conflict with any other properties stored in formData. However...
It then runs $listModel->removeTableNameFromSaveData to remove the table name and ___ and just leave the field names. This is fine so long as the shortened names are a. unique within the elements being saved (which might not be true) and b. don't conflict with any other properties used by fabrik. But I can easily see how these might NOT be unique. Indeed with common field names such as id it is quite likely for there to be a conflict.
I would recommend that:
1. The removeTableNameFromSaveData returns only placeholders.
2. The returned values are stored separately for each table and separate from other fabrik formData.
I will update this further once I have fully tracked down the problem and submitted a fix PR.
Sophist
Guest
OK - I have tracked down the problem causing blank fields and fixed it in components/com_fabrik/models/importcsv.php as follows:
from:
to:
Essentially a property used in elementModel.onStoreRow was not being set.
However, it is now duplicating row 1 into row 2 (testing only with 2 rows at present for simplicity) and I have not tracked this down yet.
from:
Code:
$formModel->formData = $aRow;
Code:
$formModel->formData = $formModel->formDataWithTableName = $aRow;However, it is now duplicating row 1 into row 2 (testing only with 2 rows at present for simplicity) and I have not tracked this down yet.
Sophist
Guest
Ok - I have now tracked down the second bug to components/com_fabrik/models/element.php getValue. This is basically storing the values in a defaults array and then reusing these next time around.
I am unclear what the purpose of the defaults array is, and so having to suck it and see - but I am unsure about how to test all the use cases which this change could affect.
I am unclear what the purpose of the defaults array is, and so having to suck it and see - but I am unsure about how to test all the use cases which this change could affect.
Sophist
Guest
Here are the two files you need to replace in components/com_fabrik/models to make importing work. (Obviously you need to remove the .txt to make them .php again.) Please apply these after installing fabrik 3.1b AND applying the latest files from GITHUB.
I have not done extensive regression testing to see if the fix I have coded has any unintended consequences. So use at your own risk.
Also, the fix to form.php is a bit of a kludge, copying a similar kludge from elsewhere in this file. The element.php getValue function needs to be looked at to see whether the use of the defaults array is actually correct (or even useful).
Uploaded as PR-746.
P
I have not done extensive regression testing to see if the fix I have coded has any unintended consequences. So use at your own risk.
Also, the fix to form.php is a bit of a kludge, copying a similar kludge from elsewhere in this file. The element.php getValue function needs to be looked at to see whether the use of the defaults array is actually correct (or even useful).
Uploaded as PR-746.
P
hardkiffeur
Member
I test it and solved my issue post here : http://fabrikar.com/forums/index.php?threads/csv-import-stuff-work.34496/
Thanks Protopia
Thanks Protopia
hardkiffeur
Member
Note : I've a link element and all informations was sav into database but when I edit raw, I've got only a "{" into label and nothing into the url. But I've this this into the database after importation :
"{\"label\":\"plan\".\"link\":\"http:\\\/\\\/www.promogerim.fr\\\/plans\"}"
And good one (like the CSV data) must feet to : {"label":"plan","link":"http:\/\/www.promogerim.fr\/plans"}
So the import add " at the start and the end of the data I was import
"{\"label\":\"plan\".\"link\":\"http:\\\/\\\/www.promogerim.fr\\\/plans\"}"
And good one (like the CSV data) must feet to : {"label":"plan","link":"http:\/\/www.promogerim.fr\/plans"}
So the import add " at the start and the end of the data I was import
Sophist
Guest
If you want to upload a sample of your CSV file, I will take a quick look. But (not yet having see the CSV file) this looks to me like several problems.
1. csv import not handling quoted strings as advertised.
2. possibly storing the URLs in the database so that they will display properly (i.e. with quoted / and \) - but IMO this is incorrect as other code may use the data in the database - it should store it native and add necessary quoting to the string as needed at the time of display.
Since my CSV files are not quoted, this is not an issue for me at present, and it will need to wait until I can find the time to look at it in more detail for me to try to debug it and find a fix.
P
1. csv import not handling quoted strings as advertised.
2. possibly storing the URLs in the database so that they will display properly (i.e. with quoted / and \) - but IMO this is incorrect as other code may use the data in the database - it should store it native and add necessary quoting to the string as needed at the time of display.
Since my CSV files are not quoted, this is not an issue for me at present, and it will need to wait until I can find the time to look at it in more detail for me to try to debug it and find a fix.
P
hardkiffeur
Member
here my csv file man, thanks for the time
hardkiffeur
Member
Arf, I made in this way for more compatibility,
1) Create one entrie under Fabrik
2) export the Csv with the sample data
3) add data to the file
4) reimport with all data
So I work only on this file
1) Create one entrie under Fabrik
2) export the Csv with the sample data
3) add data to the file
4) reimport with all data
So I work only on this file
Sophist
Guest
Well - if this is essentially data that you have saved through a form and exported it, then I am really not sure what the problem is?
I would say that the export should not have escaped \ or /, and probably should not have JSONs in it but that is just my opinion.
So without more information, I cannot understand what the problem is, nor reproduce it nor debug and fix it.
P
I would say that the export should not have escaped \ or /, and probably should not have JSONs in it but that is just my opinion.
So without more information, I cannot understand what the problem is, nor reproduce it nor debug and fix it.
P
cheesegrits
Support Gopher
Did PR746 get merged? I'm coming in to this thread a bit late, don't see 746 in the request queue.
--hugh
--hugh
Sophist
Guest
leeburstroghm
Member
well, not sure if this is related exactly. import is still really acting strange.
the fields are all off, if you are trying to import with now all the fileds in th csv, or if there are more than you need. and you are just trying to grab a couple.. the whole fields offset or shift and you end up with filed data you dont want in your existing fileds.. this is strange...
the fields are all off, if you are trying to import with now all the fileds in th csv, or if there are more than you need. and you are just trying to grab a couple.. the whole fields offset or shift and you end up with filed data you dont want in your existing fileds.. this is strange...
