Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Command example (with variable substitution) does not work #2905

Closed
bdbaddog opened this issue Jan 2, 2018 · 11 comments · Fixed by #4665
Closed

Command example (with variable substitution) does not work #2905

bdbaddog opened this issue Jan 2, 2018 · 11 comments · Fixed by #4665
Milestone

Comments

@bdbaddog
Copy link
Contributor

bdbaddog commented Jan 2, 2018

This issue was originally created at: 2013-04-29 09:01:31.
This issue was reported by: dsiroky.

dsiroky said at 2013-04-29 09:01:31

Example at the bottom of https://www.scons.org/doc/production/HTML/scons-user/c3895.html

thus

env.Command('${SOURCE.basename}.out', 'foo.in', build)

or

env.Command('${SOURCE.basename}.out', 'foo.in', "sed 's/x/y/' < $SOURCE > $TARGET")

kills the build process.

I tracked the reason, and it is an exception which is not printed, and it silently exits the scons. The exception is AttributeError 'str' object has no attribute 'basename' trying to evaluate '${SOURCE.basename}'.

It can be temporarily solved like this:

env.Command('${SOURCE.basename}.out', File('foo.in'), build)

Votes for this issue: 1.

@ajf58
Copy link
Contributor

ajf58 commented Feb 24, 2018

This is fixed in the current 3.0.1 documentation (see http://scons.org/doc/production/HTML/scons-user.html#chap-builders-commands). I suggest closing as 'fixed'.

@bdbaddog bdbaddog added this to the 3.0 milestone Feb 24, 2018
@bdbaddog
Copy link
Contributor Author

Fixed.

@bdbaddog bdbaddog added the Fixed Issue Fixed label Feb 24, 2018
@ompadu
Copy link

ompadu commented Dec 8, 2024

I'm still having this problem in 4.8.1

@mwichmann
Copy link
Collaborator

I'm still having this problem in 4.8.1

Can you include a current example?

Not sure what was "fixed" in the docs, as they continue to have the reference for basename, and don't incldue the "trick" of converting the source to a Node first.

Can you try using .filebase instead? That's actually the documented attribute for this:

https://scons.org/doc/production/HTML/scons-man.html#special_attributes

There may still be a problem in evaluation order in Command, such that something is attempted before the conversion of filename strings to Nodes.

@mwichmann
Copy link
Collaborator

A little poking around and there's a path through here that doesn't work out right. Since this issue has been closed for almost seven years, I'm going to open a new one rather than reopening this one.

@mwichmann
Copy link
Collaborator

See #4660 for the new issue.

@bdbaddog
Copy link
Contributor Author

@ompadu If you use:

env.Command('${SOURCE.filebase}.out', File('foo.in') , "echo $TARGET $SOURCE")

It should work. This should get you unstuck if you were stuck. Until such time as we can get it working as you'd expect.

@ompadu
Copy link

ompadu commented Dec 14, 2024

@ompadu If you use:

env.Command('${SOURCE.filebase}.out', File('foo.in') , "echo $TARGET $SOURCE")

It should work. This should get you unstuck if you were stuck. Until such time as we can get it working as you'd expect.

Thank you for the quick fix. It goes pass the initial error but it seems that it only processes the first item of a list:

env.Command('${SOURCE.filebase}.out', [File(file) for file in ['foo_1.in', 'foo_2.in']] , "echo $TARGET $SOURCE")

output:

echo foo_1.out foo_1.in
"foo_1.out" "foo_1.in"

@bdbaddog
Copy link
Contributor Author

That's what I'd expect, you've specified ${SOURCE.filebase}.out (Singular).
Your example likewise only had a single source.

In your updated example, are you expecting foo_1.out & foo_2.out? Are both these files processed with a single command? "echo $TARGET $SOURCE" ?
Or are you expecting it to be run multiple times? once per input source?
If the latter, then

[ env.Command('${SOURCE.filebase}.out', File(file) , "echo $TARGET $SOURCE") for file in ['foo_1.in', 'foo_2.in' ]
Should do the trick.

@mwichmann
Copy link
Collaborator

SOURCE is only the first item, by definition. SOURCES is all the sources. Same for TARGET and TARGETS. They're special cases, I guess the idea is it's so common to need SOURCES[0] and/or TARGETS[0] that way back when it was decided to include the singular forms as well.

@ompadu
Copy link

ompadu commented Dec 15, 2024

That's what I'd expect, you've specified ${SOURCE.filebase}.out (Singular). Your example likewise only had a single source.

In your updated example, are you expecting foo_1.out & foo_2.out? Are both these files processed with a single command? "echo $TARGET $SOURCE" ? Or are you expecting it to be run multiple times? once per input source? If the latter, then

[ env.Command('${SOURCE.filebase}.out', File(file) , "echo $TARGET $SOURCE") for file in ['foo_1.in', 'foo_2.in' ] Should do the trick.

The second option was the case. I was processing a list of files, each witch its own input, output and call to the external tool. Now it works how I expected. Thank you!

SOURCE is only the first item, by definition. SOURCES is all the sources. Same for TARGET and TARGETS. They're special cases, I guess the idea is it's so common to need SOURCES[0] and/or TARGETS[0] that way back when it was decided to include the singular forms as well.

Thank you, now I understand better how this aspect works!

mwichmann added a commit to mwichmann/scons that referenced this issue Dec 16, 2024
The User Guide uses an example (in the Command chapter) of making a target
name out of the source name, a special attribute, and concaetenation
with a new suffix.  The special attribute part is not something that has
been introduced. Some searching suggests it's never actually described in
the User Guide, though .file, .abspath and .srcdir are used in examples.

The attribute used, .basename, doesn't actually exist - there's a .base
(and also a .filebase. Furthermmore, the substitution suggested doesn't
work. Expansion of special variables like $SOURCE into nodes is deferred -
see the docstring of SCons.Subst.NLWrapper - so the internal expansion
ends up trying to lookup the attribute on a string, which fails with
an AttributeError.  The way the user guide entry is written, it was
not actually evaluated: it was described as an <scons_example>, but
an incomplete one, and since there was no corresponding <scons_output>
the problem was not detected.

The changes fix up the example to have it use an existing attribute
and do File() on the source, and add a sidebar to provide a bit of an
explanation so this isn't just "magic".  A subsequent example (that
I added) is dropped as it doesn't add enough value, and actually
causing the evaluation to be run is added to the final example of the
chapter - the troublesom example was ex3, and wasn't set up to ever
be executed by the doc machinery, so the failure wasn't spotted;
now it can be shown to be working correctly.

This finally fixes SCons#2905, which was closed in 2018 as having been addressed,
though the non-working example actually remained. The issue is
also mentioned in SCons#4660, which is not resolved by changing the docs -
leaving that open, at least for the time being.

Signed-off-by: Mats Wichmann <mats@linux.com>
mwichmann added a commit to mwichmann/scons that referenced this issue Dec 16, 2024
The User Guide uses an example (in the Command chapter) of making a target
name out of the source name, a special attribute, and concatenation
with a new suffix.  The special attribute part is not something that has
been introduced. Some searching suggests it's never actually described in
the User Guide, though .file, .abspath and .srcdir are used in examples.

The attribute used, .basename, doesn't actually exist - there's a
.base and a .filebase. Furthermore, the substitution suggested doesn't
work. Expansion of special variables like $SOURCE into nodes is deferred -
see the docstring of SCons.Subst.NLWrapper - so the internal expansion
ends up trying to lookup the attribute on a string, which fails with
an AttributeError.  The way the user guide entry is written, it was
not actually evaluated: it was described as an <scons_example>, but
an incomplete one, and since there was no corresponding <scons_output>
the problem was not detected.

The changes fix up the example to have it use an existing attribute
(.base) and do File() on the source, and add a sidebar to provide a
bit of an explanation so this isn't just "magic".  A subsequent example
(ex4, which I added) is dropped as it doesn't add enough value by itself,
and the final example (formerly ex5, renamed to ex3) now includes this
substitution so it's actually run by the doc machinery, and can be
seen to be working correctly.

This finally fixes SCons#2905, which was closed in 2018 as having been addressed,
though the non-working example actually remained. The issue is
also mentioned in SCons#4660, which is not resolved by changing the docs -
leaving that open, at least for the time being.

Signed-off-by: Mats Wichmann <mats@linux.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants