Calc element not using correct placeholder

[Bauer]

I have a calc element that started throwing an error. I'm not sure when this was introduced as it's been a while since I saved this form - and the calc element is set to only calc on save.

A screen snip of the error is attached.
This is the code used in the calc that is now generating the error...

$myQuery = 'SELECT m.facility_type_id, m.member_facility_id, ft.survey_participation_credits FROM master_report as m LEFT JOIN fb_facility_types as ft ON m.facility_type_id=ft.id WHERE m.user_id={fb_conglomerates___user_id_raw||0} GROUP BY m.member_facility_id';

fb_conglomerates___user_id is a 'user' element. Yet the calc is now using the user name rather than the raw value as specified in the placeholder.

 

[Bauer]

Below is a partial copy of a print_r( $this->_searchData, 1 ) data immediately after the form is submitted (in the the parseMessageForPlaceHolder function of parent.php)...

[Submit] =>
[view] => form
[fb_conglomerates___id_raw] => 74
[fb_conglomerates___user_id_raw] =>
[fb_conglomerates___membership_name_raw] => Joe Fabitz Membership 2015
[fb_conglomerates___hospital_system_raw] => Array
(
[0] => 3
)

[fb_conglomerates___landing_page_raw] => Array
(
[0] => 1
)

[fb_conglomerates___offline_filetype_raw] => Array
(
[0] => xlsx
)

As you can see [fb_conglomerates___user_id_raw] is empty (which is why it ends up using the label instead).

This element is using the user plugin set as 'Hidden'.
Regardless the label that is set, whether hidden or not, it is supposed to always return the raw value - and using it as a placeholder has been working until just very recently.

There is probably a way around this - but that isn't the point. The point is, many Fabrik users might have the user element configured like this - and use the user_id_raw placeholder - and suddenly it will stop working and produce an error.

 

[troester]

I can't replicate.
I used exactly your code (with my userid element).
Userid element set to hidden, calc set to only calc on save, no ajax calc, hidden
calc

$myQuery = 'SELECT m.facility_type_id, m.member_facility_id, ft.survey_participation_credits FROM master_report as m LEFT JOIN fb_facility_types as ft ON m.facility_type_id=ft.id WHERE m.user_id={childlist___userid_raw||0} GROUP BY m.member_facility_id';
return $myQuery ;

List view is showing

and this is also what's stored in the DB.

 

[Bauer]

Thanks for the feedback troester. And why am I not surprised that Phil's Law stike again?
I wonder if this is because the form is used in a Fabrik form module? Or is this a PHP7 thing? (I'm now using joomla 3.5 and php7 - and one thing for sure, it's not the first weird new issue to come up for me in recent days.)

Also I may have it set so the user element only updates the user_id when the row is first added. I wonder if that could be it?

If you could, please try adding
error_log( print_r( $this->_searchData, 1 ) );
at approx. line 747 in the parseMessageForPlaceHolder function of helper.php and see if you get results like mine.

What is weird is that as it goes through the loop of all of the $msg (placeholders) sent to the function from this page, $this->_searchData sometimes shows [fb_conglomerates___user_id_raw] => correctly, and sometimes it is blank (as shown in the last post).

By adding some debugging code throughout the replaceWithFormData function in helper.php, I can see the exact point where [fb_conglomerates___user_id_raw] => changes from blank to using the user name, but neither is correct and I have no idea why this suddenly started happening.

For now, I 'fixed' this by changing the placeholder to {$my->id} - but that ruins the code if I am logged in as admin trying to look at other user's data.

 

[cheesegrits]

I can't replicate this either. Tried all kinds of combinations.

At some point I'll set up another PHP7 test site, after managing to toast my last one.

-- hugh

 

[Bauer]

I still have this problem. This is a different calc element. (see attachment)

So I tried adding this code as the 1st line in the calc element...

$ftid = '{fb_member_facilities___facility_type_id_raw||0}';
error_log('element id9722 ftid is '.$ftid);

and this is what I get...
[10-Apr-2016 07:44:55 America/Denver] element id9722 ftid is Home Care & Services

 

[troester]

I really can't replicate.
What are your user element settings? Access? Update on edit? ...
Form plugins?

Which exact Fabrik version?

 

[Bauer]

I really can't replicate.
What are your user element settings? Access? Update on edit? ...
Form plugins?

Which exact Fabrik version?

I see what's happening here. This happens with any form that has databasejoin elements where the Access level is set to only allow edit when new.

So when the form is submitted in any subsequent edits, the $_POST[*_raw] value for those elements is the label ! - and that is the '_raw' value used by the calc element !

Below is the result of a print_r($data,1) echoed to the php error log at the start of the function _getV($data, $repeatCounter) in calc.php...

In the cut and paste from the error log that I included below, I added an astersik (*) ahead of every databasejoin element in the form - (there are 9). The 2 that are not editable have a triple asterisk (***).

As you can see, the ones that are editable include the correct *_raw data. But the 2 that are not editable are only included in the [Submit] array of the data, and the _raw values used for those 2 non-editable databasejoin elements is the label!

It seems like the solution would be to get the real raw values from the encrypted values that were stored by the _cryptViewOnlyElements function in ./components/com_fabriok/views/form/view.base.php ( as seen below in the $data[fabrik_vars][querystring] array ).

But that's way beyond my pay grade.

[10-Apr-2016 10:21:54 America/Denver] 71calc.php data is Array
(
    [fb_member_facilities___id] => 36
*  [fb_member_facilities___associated_parent_org] => Array
        (
            [0] => 0
        )

   [fb_member_facilities___hospital_type] => Array
        (
            [0] => 0
        )

*    [fb_member_facilities___hospital_ownership] => Array
        (
            [0] => 0
        )

    [fb_member_facilities___hospital_teaching] => Array
        (
            [0] => No
        )

    [fb_member_facilities___hospital_gross_income] => 0
    [fb_member_facilities___beds] => 0
    [fb_member_facilities___ftes] => 222.0
    [fb_member_facilities___management_options] => Array
        (
            [0] => 0
        )

    [fb_member_facilities___practice_physician_count] => 0
*    [fb_member_facilities___homehealth_services] => Array
        (
            [0] => Personal Care
            [1] => Nursing Aide Care
            [2] => Nursing Care
        )

    [fb_member_facilities___medicare_certified] => Array
        (
            [0] => 0
        )

    [fb_member_facilities___date_applied] =>
    [fb_member_facilities___mng_completed_count] => 0
    [fb_member_facilities___useable_date] =>
    [fb_member_facilities___expiration_date] =>
    [fb_member_facilities___hospital_count] => 0
    [fb_member_facilities___age_of_data] => 0
    [fb_member_facilities___key_count] => 0
    [fb_member_facilities___participation_credits] => 2
    [fb_member_facilities___mng_participation_credits] => 0
    [fb_member_facilities___membership_id] => 74
    [fb_member_facilities___hospital_system_id] => Array
        (
            [0] => Fabitz Hospital System
        )

    [fb_member_facilities___mng_locked] => Array
        (
            [0] => 0
        )

    [fb_member_facilities___locked] => Array
        (
            [0] => 0
        )

    [fb_member_facilities___last_edit] => 2016-03-28 00:00:00
    [listid] => 257
    [listref] => 257
    [rowid] => 36
    [Itemid] => 2057
    [option] => com_fabrik
    [task] => form.process
    [isMambot] =>
    [formid] => 258
    [returntoform] => 0
    [fabrik_referrer] =>
    [fabrik_ajax] => 0
    [package] => fabrik
    [packageId] =>
    [e4e2047ed302c05ae1837c9d31e7eb47] => 1
    [format] => html
    [fabrik_repeat_group] => Array
        (
            [505] => 1
        )

    [fabrik_vars] => Array
        (
            [querystring] => Array
                (
                   [fb_member_facilities___facility_type_id] => 2B51 C582A
                   [fb_member_facilities___facility_id] => 2B51 C4346 16D 4
                    [fb_member_facilities___completed_count] => 40
                    [fb_member_facilities___user_id] => 2B51 B4B4F1112
                )

        )

    [Submit] =>
    [fb_member_facilities___id_raw] => 36
***    [fb_member_facilities___facility_type_id_raw] => Array
        (
            [0] => Home Care & Services
        )

***   [fb_member_facilities___facility_id_raw] => Array
        (
            [0] => Fabitz Home Care
        )

*    [fb_member_facilities___associated_parent_org_raw] => Array
        (
            [0] => 0
        )

*    [fb_member_facilities___hospital_type_raw] => Array
        (
            [0] => 0
        )

*    [fb_member_facilities___hospital_ownership_raw] => Array
        (
            [0] => 0
        )

    [fb_member_facilities___hospital_teaching_raw] => Array
        (
            [0] => 0
        )

    [fb_member_facilities___hospital_gross_income_raw] => 0
*    [fb_member_facilities___hospice_services_raw] =>
    [fb_member_facilities___beds_raw] => 0
    [fb_member_facilities___ftes_raw] => 222.0
    [fb_member_facilities___management_options_raw] => Array
        (
            [0] => 0
        )

    [fb_member_facilities___practice_physician_count_raw] => 0
*    [fb_member_facilities___practice_specialties_raw] =>
*    [fb_member_facilities___homehealth_services_raw] => Array
        (
            [0] => 2
            [1] => 1
            [2] => 3
        )

    [fb_member_facilities___medicare_certified_raw] => Array
        (
            [0] => 0
        )

    [fb_member_facilities___identifier_raw] =>
    [fb_member_facilities___facility_order_raw] =>
    [fb_member_facilities___date_applied_raw] =>
    [fb_member_facilities___completed_count_raw] => 0
    [fb_member_facilities___mng_completed_count_raw] => 0
    [fb_member_facilities___user_id_raw] => Array
        (
            [0] => 318
        )

    [fb_member_facilities___useable_date_raw] =>
    [fb_member_facilities___expiration_date_raw] =>
    [fb_member_facilities___hospital_count_raw] => 0
    [fb_member_facilities___age_of_data_raw] => 0
    [fb_member_facilities___key_count_raw] => 0
    [fb_member_facilities___member_facility_selector_raw] =>
    [fb_member_facilities___participation_credits_raw] => 2
    [fb_member_facilities___mng_participation_credits_raw] => 0
    [fb_member_facilities___membership_id_raw] => 74
*    [fb_member_facilities___hospital_system_id_raw] => Array
        (
            [0] => 3
        )

    [fb_member_facilities___survey_link_raw] =>
    [fb_member_facilities___mng_locked_raw] => Array
        (
            [0] => 0
        )

    [fb_member_facilities___locked_raw] => Array
        (
            [0] => 0
        )

    [fb_member_facilities___last_edit_raw] => 2016-03-28 00:00:00
    [__pk_val] => 36
    [fabrikdebug] => 1
    [language] => en-GB
    [view] => form
    [lang] => en-GB
    [fb_member_facilities___facility_type_id] => Array
        (
            [0] => Home Care & Services
        )

    [fb_member_facilities___facility_id] => Array
        (
            [0] => Fabitz Home Care
        )

    [fb_member_facilities___completed_count] => 0
    [fb_member_facilities___user_id] => Array
        (
            [0] => 318
        )

    [fb_member_facilities___hospice_services] =>
    [fb_member_facilities___practice_specialties] =>
    [fb_member_facilities___identifier] =>
    [fb_member_facilities___facility_order] =>
    [fb_member_facilities___member_facility_selector] =>
    [fb_member_facilities___survey_link] =>
)

 

[troester]

Still the question about your exact Fabrik version (as also with user element "form (edit)"=nobody on my site the calc is getting the user id).

 

[Bauer]

Still the question about your exact Fabrik version (as also with user element "form (edit)"=nobody on my site the calc is getting the user id).

Sorry. Joomla 3.5.1 - Fabrik 3.4.3 (w/latest github) - PHP 7.0.5
The user element plugin is not the datbasejoin element plugin. Please DO test.

 

[Bauer]

Hmm, sorry. OK it is a 'Phil's Law' thing after all. I have another form with a similar situation and that one works correctly. It must be something I'm doing - but I've been days at trying to figure out just what is so different with this form.

 

[Bauer]

I was wrong about that - I can duplicate in any form.
The only reason it didn't give me an error is because I wasn't referencing the readonly dbjoin raw value anywhere in a calc element in that same group.
That's all you have to do to duplicate.

E.g. Take these steps to duplicate...
Create a dbjoin element that can only be edited when New.
Add a new row to that list.
Select a dbjoin value in the form and save.

Then, assuming the dbj element is mytable___mydbj....
Create a calc element in that same group and add these lines
$test = '{mytable___mydbj_raw||0}';
error_log('mytable___mydbj raw value is '.$test);

Now re-edit that same row that you just added in the list and save.
Then look at the php error log.

 

[cheesegrits]

It seems like the solution would be to get the real raw values from the encrypted values that were stored by the _cryptViewOnlyElements function

Hmm. I remember fixing that exact issue a while back ... lessee ... yeah, in the main form model validate() ...

        /* $$$ hugh - add in any encrypted stuff, in case we fail validation ...
        * otherwise it won't be in $data when we rebuild the page.
        * Need to do it here, so _raw fields get added in the next chunk 'o' code.
        */
        $this->addEncrytedVarsToArray($post);

        // $$$ hugh - moved this to after addEncryptedVarsToArray(), so read only data is
        // available to things like calcs running in preProcess phase.
        $this->callElementPreProcess();

... as you can see from the comment, I ran across this exact issue before. Calc's are done in the element preProcess phase. However, I suspect it wasn't with a join element, so there's probably some additional hackery needs doing to ensure the value and label go in the appropriate places.

Looking at it now.

-- hugh

 

[cheesegrits]

Well, guess what. I still can't replicate this.

The encrypted raw value gets correctly inserted into the posted data in addEncryptedVarsToArray(), then in the calc preProcess(), the swapValuesForLabels() does it's thing, and sets the non _raw to the label, and leaves the raw as raw. And my calc ...

return 'event: {fab_main_test___event}, event raw: {fab_main_test___event_raw}';

correctly gives me ...

event: another test, event raw: 5

... where fab_main_test___event is a join, set to "nobody" for edit.

-- hugh

 

[cheesegrits]

That's on submit, tho.

Haven't tried it on display ("only calc on save: NO"). May well fail there.

And I can pretty much guarantee it'd fail on AJAX, and that's a tough fix.

-- hugh

 

[Bauer]

All I know is my head is spinning. (I need to make me some dinner)
Using my example, with the calc showing the supposed raw value of the dbj in the error log, this is what I get in the error log.

This is a basic test list - no repeat groups - one dbjoin (test_list___facility_type) - and one calc element...

[10-Apr-2016 14:32:53 America/Denver] test_list___facility_type raw is 2
That is what is in the log right after the form is loaded - which is correct!

Then after I save the form I get these FOUR new lines added in the log...
[10-Apr-2016 14:35:58 America/Denver] test_list___facility_type raw is Extended Care Nursing-Rehab
[10-Apr-2016 14:35:58 America/Denver] test_list___facility_type raw is 2
[10-Apr-2016 14:35:58 America/Denver] test_list___facility_type raw is 2
[10-Apr-2016 14:35:59 America/Denver] test_list___facility_type raw is

It looks like the first time the calc element is "looked at" on submit - it is using the label as the value (which is why my query using it blows up).
Then it 'fixes itself' for 2 cycles - then ends up returning an empty, null, or false value.
Go figure - my head's ready to explode.

But something doesn't seem right - that the same code gets run 4 times after form submit.

Edit addendum: This calc is set with 'Calc on load' and 'Only calc on Save' both set to No.

 

[troester]

I think I've seen this "multiple running calc" before, too.

I can confirm that the calc is running 3 times on form save (doesn't matter if "only calc on save"=no or yes) but in my log it's always showing correct {element} and {element_raw} (tested for a user element + a dbjoin element, both with form(edit)=nobody).
I can't replicate your "label" - but I'm not on PHP7.

"[10-Apr-2016 14:35:59 America/Denver] test_list___facility_type raw is"
I assume this is the log of your list view. The {element||default} syntax is not working in list view, seems in this case it's an "unknown" element and therefor empty.

 

[Bauer]

Thanks troester.
So you agree, that there seems to be some circumstances (only with a calc element referencing a dbjoin element set as readonly after add?) where there seems to be some loop or duplication of the eval - where the calc eval is run multiple times before the REAL raw value can be determined?

Now the big question, to verify/duplicate the problem... Are you using a dbjoin element in the list where the dbjoin element Access is set to 'Special' or 'Nobody' for edit (can only be edited when new/Add) ? That is what seems to cause this problem, where the dbjoin label is initially being used instead of the raw value (on save of subsequent edits of the form).

That's why I remarked, after your reply where you mentioned that you where using a user element and it was showing the correct raw value, "The user element is not a dbjoin element".
I wasn't just being sarcastic there - I'm just trying top pinpoint and verify that this is only happening with a dbjoin element that is configured to only be editable when add/new.

 

[troester]

As I've said I see the multiple calc but I can't replicate your "label" issue:

my log it's always showing correct {element} and {element_raw} (tested for a user element + a dbjoin element, both with form(edit)=nobody)

 

[Bauer]

Sorry, I must have looked right over the end of that. I'm almost hoping it's a php7 issue then (though I can't see why that would be), because I sure can't figure out why this seems to only be happening for me.