distributed collective mind
473 stories
·
3 followers

Testing on the Toilet: Tests Too DRY? Make Them DAMP!

1 Comment
This article was adapted from a Google Testing on the Toilet (TotT) episode. You can download a printer-friendly version of this TotT episode and post it in your office.

By Derek Snyder and Erik Kuefler

The test below follows the DRY principle (“Don’t Repeat Yourself”), a best practice that encourages code reuse rather than duplication, e.g., by extracting helper methods or by using loops. But is it a well-written test?
def setUp(self):
self.users = [User('alice'), User('bob')] # This field can be reused across tests.
self.forum = Forum()

def testCanRegisterMultipleUsers(self):
self._RegisterAllUsers()
for user in self.users: # Use a for-loop to verify that all users are registered.
self.assertTrue(self.forum.HasRegisteredUser(user))

def _RegisterAllUsers(self): # This method can be reused across tests.
for user in self.users:
self.forum.Register(user)

While the test body above is concise, the reader needs to do some mental computation to understand it, e.g., by following the flow of self.users from setUp() through _RegisterAllUsers(). Since tests don't have tests, it should be easy for humans to manually inspect them for correctness, even at the expense of greater code duplication. This means that the DRY principle often isn’t a good fit for unit tests, even though it is a best practice for production code.

In tests we can use the DAMP principle (“Descriptive and Meaningful Phrases”), which emphasizes readability over uniqueness. Applying this principle can introduce code redundancy (e.g., by repeating similar code), but it makes tests more obviously correct. Let’s add some DAMP-ness to the above test:

def setUp(self):
self.forum = Forum()

def testCanRegisterMultipleUsers(self):
# Create the users in the test instead of relying on users created in setUp.
user1 = User('alice')
user2 = User('bob')

# Register the users in the test instead of in a helper method, and don't use a for-loop.
self.forum.Register(user1)
self.forum.Register(user2)

# Assert each user individually instead of using a for-loop.
self.assertTrue(self.forum.HasRegisteredUser(user1))
self.assertTrue(self.forum.HasRegisteredUser(user2))

Note that the DRY principle is still relevant in tests; for example, using a helper function for creating value objects can increase clarity by removing redundant details from the test body. Ideally, test code should be both readable and unique, but sometimes there’s a trade-off. When writing unit tests and faced with a choice between the DRY and DAMP principles, lean more heavily toward DAMP.

Read the whole story
sness
26 days ago
reply
DAMP principle (“Descriptive and Meaningful Phrases”)
milky way
Share this story
Delete

Implementing pyLucid

1 Share

In a companion post I give an overview of pyLucid, a version of Lucid close to that found in the Lucid book and implemented in Python. Here is an overview of the implementation.

The first stage is parsing. I used a not-very–well-known mixture of recursive descent and operator precedence (which I reinvented independently). Instead of separate functions for every level of precedence, it has a single function parametrized with the precedence level. For error checking it has a function expect parametrized with a lexical token. If the token doesn’t show up as expected, an error message is produced along with the first five tokens that showed up instead. This is usually enough to pinpoint the syntactic error.

The parser produces not a parse tree but objects in a multi-sorted abstract data type that models the various syntactic categories. For example it may produce a where (clause) that has a subject and a body, which is a list of definitions each of which has a rhs and a lhs and so on. The abstract data type has predicates, constructors, and destructors. For example WhereP(w) tests if w is a where clause, WhereSubject(w) returns its subject, and WhereC(s,b) returns a clause with subject s and body b.

The abstract data type is implements with POP2 constructs but this fact, and the implementation details, are a closely guarded secret found only in one file. In turn the POP2 data objects are implemented by Python constructs and the details are found only in another file. And this file also implements POP2 I/O which, as I indicated, doubles as the pyLucid lexxer.

Early on I realized that it was much more convenient to have a single destructor that returns components (Python allows multiple returns). Thus instead of writing

s = WhereSubject(w)
b – WhereBody(w)

you write

s, b = WhereD(w)

Once the program passes the syntax check and is turned into a where clause the  semantic checking begins. We verify that the where bodies have at most one definition per variable and that the formal parameters of functions are all distinct. We check that function arities are consistent (always have the same  number of parameters) and finally that every variable (besides input) has a relevant definition.

Once the checking is done we proceed to transform the program to a form that can be directly evaluated. The first step is a systematic renaming of variables so that every logically different variable has a different name.  For example  if the   program is

X+I
where
X = f(Y);
f(X) = X+Y*I
where
Y = 7;
I = 2;
end;
I=9;
end;

[sorry, I haven’t figured out indenting] the result of renaming is

X+I
where
X = f(Y);
f(X_01) = X_01+Y_01*I
where
Y_01 = 7;
end;
I = 9;
Y = 3;
end;

The crucial points are (1) the transformed program is still a pyLucid program; and (2) it’s semantically equivalent to the original. This will be true at each stage, so we will end  up with a very simple (but long) program equivalent to the original source.

At several points we will ‘dissolve’ where clauses by simply dumping their definitions into the enclosing set of definitions. Thus

A+B
where
A = 9;
B = 2 * A
where
A = 7;
end;
end;

is renamed to

A+B*3
where
A = 9;
B = 2 * A_01
where
A_01 = 7;
end;
end;

then the inner where clause can be dissolved giving

A+B*3
where
A = 9;
B = 2 * A_01
A_01 = 7;
end;

The most important transformation is  the elimination of user defined functions basically by transforming function calls into branching  time iteration (I’ll save the details for another post). Then the program has only zero-order (individual) variables and the rest of the where clauses can be dissolved.

Finally by introducing extra variables we turn compound expressions into ‘atomic’ expressions (only one operation) and the whole program is now a set of atomic equations: one individual variable on the left equated to an expression consisting of an operation applied to individual variables (or a literal/constant).

The program above becomes

output = A+V_00;
V_00 = B*V01;
V_01 = 3;
A = 9;
B = V_02 * A_10;
V_02 = 2;
A_01 = 7;

and it is ready for evaluation.

We can consider the atomic equations form of the program as the analog of assembly language. The output of the compiler is the assembly language and nothing else, and that is the input to the interpreter/evaluator. Thus they communicate with a human-readable text file.

Eduction  of the assembly language is straight forward. The evaluator demands values of particular variables for particular space and time coordinates. These coordinates are stored in ‘registers’ rather than passed around as parameters.

For example, if we demand A and it it defined as

A = B+C

then we evaluate B and C without changing registers and return the sum of these values.

If

A = next B

we bump the time register by one, evaluate B, then decrement the time register (restoring its initial value).

If

A = B sby C

we examine the space register. If  it’s 0 we evaluate B, otherwise we decrement it, evaluate C, then increment it.

And so on (check out the eduction blog post).

There are actually more registers than just time and space.  Our treatment of nesting implies a hierarchy (usually very shallow) of extra time parameters, one for each current level of nesting. There are another two registers specifying the current “place” parameter (place in the function calling tree). The compiler introduces special operators that explicitly manipulate these coordinates.

We have to save computed values in a cache to avoid recomputing them. There is so much memory on even a laptop these days that for the time being we didn’t implement the retirement plan to periodically thin out the cache.

We did, however, find a simple strategy that greatly reduces the size of the cache – we don’t cache variables that are the result of data operations. Thus if A=B+C, we don’t cache A because it can be recomputed B and C.

While exercising the interpreter we discovered an interesting phenomenon: many if not most cached values are never retrieved. We keep track of variables for which at least  one value is retrieved. In practice the new V- variables are not retrieved and the same holds for most of the original program variables. Typically even with a larger program only a half dozen or so program variables actually benefit from caching. If we  could understand this phenomenon better we could save a lot of memory.

 

 

 

 

 

 

 

 

 

 

 

 

 

 



Read the whole story
sness
26 days ago
reply
milky way
Share this story
Delete

The Ideal Place

1 Comment

“I’ve been searching for years for the ideal place. And I’ve come to the realization that the only way to find it is to be it.”
— Alan Watts

Read the whole story
sness
26 days ago
reply
true
milky way
Share this story
Delete

Walk Beside Me

2 Shares

“Don’t walk behind me; I may not lead. Don’t walk in front of me; I may not follow. Just walk beside me and be my friend”.
Albert Camus

Read the whole story
sness
26 days ago
reply
milky way
Share this story
Delete

Link Pack

1 Comment

(via)

How the X-Ray was invented.

What not to do on your work computer.

– Love this list by the DO Lectures: 12 Books To Help You Write Better

The Decade the Internet Lost Its Joy. What began as cheerful anarchy was devoured by vulture capital and ruthless consolidation

This made me laugh so hard. Watch the video.

– Our new Designer at Tattly, Minjoo, created an incredibly sweet calendar for 2020.

– This. Right in the heart. The hug at the end with his mom completely pushed me over the edge.

Billie Eilish Carpool Karaoke. Fun.

A table on wheels, looking for conversations.

Lego Frida Kahlo

Building a More Honest Internet (via Dense Discovery)

Visualizations that make no sense. It keeps on giving.

– Looking for a last minute gift that keeps on giving? Tattly Subscriptions!

– A big thank you to Baleen for sponsoring my blog and RSS Feed this week.

Read the whole story
sness
26 days ago
reply
amazing cats
milky way
Share this story
Delete

Australia is using sound to monitor its wildlife and ecosystem

1 Comment

Listen to the Earth – no, literally. Australia has a world’s-first project to map the voice of its wildlife, which will make a “galaxy of sounds” that can be heard like the ecosystem’s heartbeat.

The galaxy metaphor isn’t just poetic. Scientists already observe the cosmos through radio signal, optical imaging, and other techniques. A network of hundreds of solar-powered audio recorders will do the same across Australia.

Those sensors themselves will be familiar to electronic composers and field recordists. They use microphones and standard SD cards, but operate entirely on solar power – a necessity for remote locations. The recordings are continuous, and spread through the vast country, across some one hundred sites and 400 sensors covering “desert, grassland, shrublands and temperate, subtropical and tropical forests.”

From a scientific standpoint, this is already a big deal, because it can look at geography and time comprehensively, instead of in little pieces that can rob recordings of context and meaning. That means seasonal change, or even changes over the five year mission, will appear in the recordings.

It’s also significant that the project, and the vast amounts of sound data it produces, will be available to the public. That hints at both educational opportunities and crowd-sourced analysis – especially since citizen Australians may be on the ground spread through these hundred locations. The whole project will keep its data online and accessible. It’s a welcome change in talking about big data, from government surveillance and corporate control to actually harnessing that power to provide access to the public and better awareness of the ecosystem on which we depend.

Audio recording is less intrusive and far more comprehensive (in time and space) than conventional methods. And there are real, practical possibilities, as ABC News in Australia reports from Queensland:

[Biology Professor Lin Schwarzkopf, James Cook University] had also mapped the noise of the aggressive Indian myna birds and the rare black-throated finch, which environmentalists warned was a species under threat from the Adani coal mine in central Queensland.

“It’s useful because we can then look at things like species decline so we can understand when they are disappearing, like the black-throated finch. Over time we can listen for their calls and see if they are still there,” she said.

Acoustic observatory will record ‘galaxy of sounds’ to help scientists monitor Australian wildlife? [ABC News]

In the case of the Australian project, there are various partners including universities, parks and government, owners, indigenous partners, and advocacy groups and NGOs.

Non-scientist Australians can get in on making observations, too, as in this interface. Check the project blog for more on how this works.

This strikes me as an opportunity for composers and musicians to partner with scientists, too. Whereas music once made primitive mimicry of bird songs, now it seems musicians possess a technological and creative/cultural skillset to both capture more data and express its significance to the broader public and society. So let’s get on it, shall we? (Hey, Europe for instance is a lot smaller and more manageable, for example…)

More on the Australian Acoustic Observatory

The acoustic recorders are made by an Australian company – https://frontierlabs.com.au/bioacoustics.html

The post Australia is using sound to monitor its wildlife and ecosystem appeared first on CDM Create Digital Music.

Read the whole story
sness
30 days ago
reply
this is super cool! Passive Acoustic Monitoring, that is what I do for my volunteer work (https://github.com/openmir/openmir)
milky way
Share this story
Delete
Next Page of Stories