cache task.to_str_params() to speedup Task initialization

[Hi-king]

Summary:

During the initialization process of gokart.TaskOnKart, especially with deeply nested TaskInstanceParameter, the repeated calls to to_str_params() caused significant performance degradation. To address this issue, we introduced a caching mechanism for function calls, which has significantly accelerated the initialization process.

Detailed Behavior:

The following actions occur during initialization:

1. to_str_params() is called during the task's init.
2. Inside to_str_params(), the parameter's serialize method is executed.
3. Within the TaskInstanceParameter's serialize, to_str_params() is called recursively.

Changes:

Implemented a caching mechanism for the to_str_params() function.
Cached the string representations of parameters once computed to prevent recalculations for the same parameters.
This change substantially reduces the time complexity from O(n^2), thereby shortening the initialization time.
Impact:
This modification affects all tasks inheriting from gokart.TaskOnKart, particularly those with complex dependencies, improving the initialization performance significantly.

How to Reproduce

class HelloWorldTask(gokart.TaskOnKart):
    _cache_key_date = luigi.DateMinuteParameter(default=datetime.datetime.now())
    
    def run(self):
        self.dump("Hello, World!")

class RecursiveTask(gokart.TaskOnKart):
    dep = gokart.TaskInstanceParameter()

    def requires(self):
        return self.dep

    def run(self):
        self.dump(self.dep.load())

def create_recursive_task(n):
    if n == 0:
        return HelloWorldTask()
    return RecursiveTask(dep=create_recursive_task(n - 1))

%time create_recursive_task(90)

Prior to this change, this sample code took 21.9 seconds to execute. With the new caching mechanism, it now takes only 1.15 seconds, marking a significant improvement in performance.

TBA

need some tests

[hiro-o918]

Nice contribution!
please fix CI errors

[Hi-king]

@hiro-o918 Thanks! I've fixed the CI errors and also added some tests!

[hirosassa]

put just a nits comment

[hirosassa]

LGTM

[kitagry]

I don't understand why this condition is necessary.

[Hi-king]

@kitagry This intend to cache only when called from init.

There're two solutions

• only cache default parameters: simple impl. , but specific use
• cache all patterns of parameters: general use, but a little bit comprecated impl.

[kitagry]

I prefer to name the condition for your intention. For example

init_condition = only_significant and (not only_public)
if !init_condition:
    return super().to_str_params(only_significant, only_public)

[Hi-king]

@kitagry Thanks! I've introduced _called_with_default_args :)

[kitagry]

[ask] default only_public is False. Is it correct?

[kitagry]

maybe?

Suggested change

	_called_with_default_args: bool = only_public and (not only_significant)
	_called_with_default_args: bool = (not only_public) and (not only_significant)

[Hi-king]

@kitagry @hiro-o918 @hirosassa

After discussing with @kitagry, we concluded that the code's intent was somewhat unclear. Therefore, we simplified it by switching to simply using functools.lru_cache .
Could you please give it a quick review and merge it? Thanks!

[hirosassa]

Looks clear and simple. I agree with this change.