HyperCard Mailing List

[HC] Report Break Sections
(Msg 1 of 11)
crrrrrrris <crrrrrrris@[redacted].com>
Monday, 02-Nov-2015 06:01 GMT
Hey friends,

I'm, once again, trying to retire my HC based accounting app that has
served so well for ... OMG 25 years.

BTW we run on 2 xServe G5s (2004), 2 PB G4s (2005) and a fleet of G4
MacMinis (2006) running 10.3.9 which is nostalgic but aging!

Every time I work on making this transition, though, i get stuck on
Reports. It's not the creation of the report file but rather the logic
behind what 9to5 Software's Reports called Break sections. Every time I
sit down to write my own code, I feel like I slept through one too many
lectures.

I figure someone else has encountered this while doing HC transitions
and might have some suggestions on how to approach it or what I should
be reading.

Thank you !
[HC] Report Break Sections
(Msg 2 of 11)
selander <selander@[redacted].jp>
Monday, 02-Nov-2015 10:55 GMT
Hi Cris,

What 9to5 calls breaks, most other software calls sub-totals.

What are you using to re-write the system in? Livecode? Sql?
Filemaker?

Tim Selander
Tokyo, Japan


On 11/2/15, 15:01, CR crrrrrrris@[redacted].com[HyperCard] wrote:
>
> Hey friends,
>
> I'm, once again, trying to retire my HC based accounting app
> that has
> served so well for ... OMG 25 years.
>
> BTW we run on 2 xServe G5s (2004), 2 PB G4s (2005) and a fleet
> of G4
> MacMinis (2006) running 10.3.9 which is nostalgic but aging!
>
> Every time I work on making this transition, though, i get
> stuck on
> Reports. It's not the creation of the report file but rather
> the logic
> behind what 9to5 Software's Reports called Break sections.
> Every time I
> sit down to write my own code, I feel like I slept through one
> too many
> lectures.
>
> I figure someone else has encountered this while doing HC
> transitions
> and might have some suggestions on how to approach it or what I
> should
> be reading.
>
> Thank you !
>
>
>
[HC] Report Break Sections
(Msg 3 of 11)
crrrrrrris <crrrrrrris@[redacted].com>
Tuesday, 03-Nov-2015 19:18 GMT
Good point about sub-totals. Thanks.

LiveCode is the choice. Since reporting is an output only function, and i think the toughest nut to crack, the plan was to start there and write LC routines that pull form the HC stacks.

I guess what I am reaching for is an approach such that I don't have to rewrite the sub-total/break section aspect of every report. So maybe this sort of approach (pardon the psudo code) ...

on makeReportData
loadParameters -- sort hierarchy, filter list, subtotal trigger list,
loadDataSet
sortDataSet
repeat for every somethingArather
filtering usingMyFilterList
subtotaling usingMySubTotalTriggerList
end repeat
end makeReportData

And next would be to write some flexible filter and subtotal routines that can apply my parameters. This has been helpful ... thanks!

I am certainly open to correction and other approaches !!!
[HC] Report Break Sections
(Msg 4 of 11)
selander <selander@[redacted].jp>
Thursday, 05-Nov-2015 23:58 GMT
Hi,

I'm trying to migrate our in-house stuff to a combo of LC and
SQL. SQL is handling the subtotals for me pretty well.

But your pseudo code is really helpful. For what's not in SQL
yet, I've been doing a lot of re-writing routines for each
report's subtotals. Never thought of trying to make a catchall
function for it! Also, I have been mentally stuck in HC's method
of repeat for x = 1 to the number of lines of a variable for
going through a list a data. For some reason, LC seems WAY SLOWER
than HC for this if the data is large. I just recently discover
repeat for each line thisLine is much faster. But I bet your idea
of filtering to subsets of the data would be faster and easier still!

Two great hints. Thanks.

Tim Selander
Tokyo, Japan


On 11/4/15, 4:18, crrrrrrris@[redacted].com[HyperCard] wrote:
>
> Good point about sub-totals. Thanks.
>
> LiveCode is the choice. Since reporting is an output only
> function, and i think the toughest nut to crack, the plan was
> to start there and write LC routines that pull form the HC stacks.
>
> I guess what I am reaching for is an approach such that I don't
> have to rewrite the sub-total/break section aspect of every
> report. So maybe this sort of approach (pardon the psudo code) ...
>
> on makeReportData
> loadParameters -- sort hierarchy, filter list, subtotal
> trigger list,
> loadDataSet
> sortDataSet
> repeat for every somethingArather
> filtering usingMyFilterList
> subtotaling usingMySubTotalTriggerList
> end repeat
> end makeReportData
>
> And next would be to write some flexible filter and subtotal
> routines that can apply my parameters. This has been helpful
> ... thanks!
>
> I am certainly open to correction and other approaches !!!
>
>
[HC] Report Break Sections
(Msg 5 of 11)
crrrrrrris <crrrrrrris@[redacted].com>
Friday, 06-Nov-2015 17:24 GMT
Ah yes the same proverbial boat!

I have found the same with regards to LC text processing speed. Arrays also help if you can take the plunge there. Also, if possible, it would be better to filter outside the repeat.


Anyways, I was walking down the stairs at home when it hit me that my 'subtotaling' function might have to somehow take into account the idea of nesting. In Reports each break section was trigger by the corresponding sort section which created a hierarchy within the breaks.


For example, if i am sorting Manager/Client/Project with corresponding break sections for each then simply testing for all the things in my break list will cause problems.


Using psudo data with my psudo code wherein A, B, ... are Mgrs, Clients, Projects, my data set sorts to...
ABC ... [other data also being in each of the rows]
ADC
BAC
CAD
DBA


Line by line ...
1 no break
2 breaks at column 2, B changed to D
3 breaks at column 1, A changed to B
4 breaks at column 3, C changed to D
4 breaks at column 1, B changed to C
5 breaks everywhere as everything changed


Assuming each break led to a subtotal then the line 4 break is problematic because it broke twice and there was nothing to subtotal. I guess the solution is to skip subtotals of just one row.


I also realized that I should step back from thinking in terms of subtotals because I also need things like page breaks. Just like Reports did !



[HC] Report Break Sections
(Msg 6 of 11)
crrrrrrris <crrrrrrris@[redacted].com>
Friday, 06-Nov-2015 17:26 GMT
Also, I think a break table of some sort needs to be kept tracking when the last break occurred. That way when one gets to a new break one knows how far back to go in calculating a subtotal.
[HC] Report Break Sections
(Msg 7 of 11)
Ambassador <Ambassador@[redacted].com>
Friday, 06-Nov-2015 19:25 GMT
> LC seems WAY SLOWER than HC for this if the data is large.

In my own comparative benchmarks LC i generally faster than HC, sometimes by a significant margin, when measuring equivalent tasks. One notable exception is the find command, and there may be others, but LC's architecture is in some ways more restrictive, giving it a shorter lookup table for token parsing.


That said, the challenge is to find truly equivalent tasks. HC is architected with 1-bit graphics, handles only ASCII, and runs only on Mac Classic, whereas LC buffers at true display color depth, handles IS-8859-1 in older versions and Unicode in v7 and later, and runs on OS X, which among other things has 16 compositing layers.


If you have code that runs in both HC and LC it would be helpful to post it here so we can determine why it may be slower. There are certainly ways to speed it up beyond HC's options since LC has arrays and other language features not found in HC, but even with equivalent code it be worthwhile seeing how performance differences play out.
[HC] Report Break Sections
(Msg 8 of 11)
scott <scott@[redacted].com>
Saturday, 07-Nov-2015 16:24 GMT
---In HyperCard-Mailing-List, <Ambassador@...> wrote :

> LC seems WAY SLOWER than HC for this if the data is large.

> In my own comparative benchmarks LC i generally faster than HC, sometimes by a significant margin, when measuring equivalent tasks. One notable exception is the find command, and there may be others, but LC's architecture is in some ways more restrictive, giving it a shorter lookup table for token parsing.


Actually this was true of MetaCard, but is no longer true of LiveCode which is now substantially slower than MetaCard was. LC 7 and 8 are on average 3 times slower than the last release of MetaCard, and this applies to everything I measured, repeat loops, function calls, math operations, chunk expressions, and file I/O. Putting text into a field is a whopping 15 times slower in LiveCode 7.1 than in MetaCard 2.5. I looked up my old benchmarks (posted to this list in 1998!) and so can confirm that in general LiveCode *is* slower than HyperCard.


> That said, the challenge is to find truly equivalent tasks. HC is architected with 1-bit graphics, handles only ASCII, and runs only on Mac Classic, whereas LC buffers at true display color depth, handles IS-8859-1 in older versions and Unicode in v7 and later, and runs on OS X, which among other things has 16 compositing layers.


But these all have nothing to do with the kind of script execution performance at issue here. Unicode support *might* be a partial explanation, but it doesn't explain why function calls, math ops, and repeat loops are so much slower in LC. And I've looked at the source and can confirm that Unicode support in LC 7.1 is almost completely unoptimized and is littered with the same kinds of unnecessary string copying that I pointed out must be happening in HyperCard back in 1998 (i.e., it's an explanation, but not really an excuse).


> If you have code that runs in both HC and LC it would be helpful to post it here so we can determine why it may be slower. There are certainly ways to speed it up beyond HC's options since LC has arrays and other language features not found in HC, but even with equivalent code it be worthwhile seeing how performance differences play out.


This is key: Dealing with large data sets without using "repeat for each" and/or "replace" and/or "split" is working with one hand tied behind your back. Of course IMHO working without a profiler (which MetaCard had but RR/LC for some inexplicable reason never did) is in the same category. It does explain why LC is so slow, though, both in the engine and in the IDE: They apparently don't know what a profiler is, or at least don't see the value in using them.
Regards,
Scott


[HC] Report Break Sections
(Msg 9 of 11)
ka4hjh <ka4hjh@[redacted].com>
Saturday, 07-Nov-2015 18:49 GMT
On Nov 7, 2015, at 11:24 AM, scott@[redacted].com[HyperCard] <HyperCard-Mailing-List> wrote:

> Of course IMHO working without a profiler (which MetaCard had but RR/LC for some inexplicable reason never did) is in the same category. It does explain why LC is so slow, though, both in the engine and in the IDE: They apparently don't know what a profiler is, or at least don't see the value in using them.

I dont know what one is, either. Please explain.


Terry Bowman, KA4HJH
"The Mac Doctor"



[HC] Report Break Sections
(Msg 10 of 11)
jacque <jacque@[redacted].com>
Saturday, 07-Nov-2015 18:58 GMT
On 11/6/2015 11:24 AM, crrrrrrris@[redacted].com[HyperCard] wrote:

> I also realized that I should step back from thinking in terms of
> subtotals because I also need things like page breaks. Just like
> Reports did !

"print break" starts a new page.

I think your topic would get a lot more answers on the LC forums or
mailing list.

--
Jacqueline Landman Gay | jacque@[redacted].comHyperActive Software | http://www.hyperactivesw.com
[HC] Report Break Sections
(Msg 11 of 11)
scott <scott@[redacted].com>
Saturday, 07-Nov-2015 19:50 GMT
---In HyperCard-Mailing-List, <ka4hjh@...> wrote :

On Nov 7, 2015, at 11:24 AM, scott@... mailto:scott@... [HyperCard] <HyperCard-Mailing-List mailto:HyperCard-Mailing-List> wrote:

> Of course IMHO working without a profiler (which MetaCard had but RR/LC for some inexplicable reason never did) is in the same category. It does explain why LC is so slow, though, both in the engine and in the IDE: They apparently don't know what a profiler is, or at least don't see the value in using them.

> I don’t know what one is, either. Please explain.

It's a tool that shows you how much time is spent on each line in your program. In MetaCard it was part of the script editor. You turn it on, run your script one or more times, and it puts a number before each line of the script that you can use to determine which statements/functions are taking the most time and so would be good candidates for optimization. After using for awhile you eventually start coding more efficiently in general because you *learn* which things are fast and which aren't.


Unfortunately for LC it's actually scripted which means that it too would run 3 times slower, significantly reducing its utility.


They're also a standard component of C++ development systems that I used a lot when developing the engine. LC clearly doesn't or they wouldn't have taken a lot of the stuff I wrote as in-line functions and turned them into (sometimes deeply) nested function calls. Besides making it a lot slower, this "refactoring" has made the engine much larger and IMHO much harder to maintain.
Regards,
Scott

> Terry Bowman, KA4HJH
> "The Mac Doctor"

HyperCard® and HyperTalk™ remain trademarks of Apple, Inc.; other trademarked products and terms mentioned in this archive are the property of their respective trademark holders. Individual messages remain the intellectual property of their respective authors.