tag:blogger.com,1999:blog-35195396944967020602024-03-05T16:21:21.948-06:00Rich Dammkoehler's Mini BlogThis is a mini-blog. I'm working to find a compromise between a tweet and a lengthy essay. I find it difficult to complete longer documents because of an obsession with perfection. So this little experiment is to see if I can create a blog of mini articles. Herein I will talk about many technical things generally related to software development and Agile practices.IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.comBlogger120125tag:blogger.com,1999:blog-3519539694496702060.post-78521579980404608882023-03-02T17:37:00.002-06:002023-03-02T17:37:33.240-06:00Fixing It, and other uses for Duct Tape<p style="text-align: center;"><span style="font-family: arial;"> "If you can't fix it with duct tape, you didn't use enough duct tape"</span></p><p style="text-align: center;"><span style="font-family: arial;">-- unknown</span></p><p style="text-align: center;"><span style="font-family: arial;">"The problem is not the problem. The problem is your attitude about the problem"</span></p><p style="text-align: center;"><span style="font-family: arial;">-- Capt. Jack Sparrow</span></p><p style="text-align: left;"><span style="font-family: arial;">I've been thinking about fixing things lately. Like the general process of fixing things, not a specific list of things that need fixing (but I have one of those too). I thought I could share some of my thoughts on fixing things here.</span></p><h3 style="text-align: left;"><span style="font-family: arial;">Don't Fix Symptoms</span></h3><div><span style="font-family: arial;">A lot of times I see people trying to fix complicated problems by attacking the symptoms of the problem. Rather than pausing a moment and understanding the context of a problem they just tape over whatever it is and move on. This leads to layers of tape. Each layer of tape has its own consequences. And eventually, entropy catches up with you and you can't fix anything. You are too busy fixing your fixes. You should be fixing core problems.</span></div><h3 style="text-align: left;"><span style="font-family: arial;">Speculation Is Risk</span></h3><div><span style="font-family: arial;">Many times the reason we're patching over symptoms is due to an incomplete understanding of the problem we see. That is, we think we know the cause of our problem and we move forward. Being prone to action and probably a little overconfident we fix what we think are core problems but we're just covering up another symptom. Speculating about the cause of a problem is risky. It's better to know the cause of the problem.</span></div><h3 style="text-align: left;"><span style="font-family: arial;">Core Problems</span></h3><div><span style="font-family: arial;">How do you know you are fixing a core problem? </span></div><div><span style="font-family: arial;"><br /></span></div><div><span style="font-family: arial;">You can't find another cause. That is, given a problem you are not able to identify another cause. This is challenging. There are a trillion moving parts in software systems, and identifying the broken one is challenging. All of us have spent hours searching the internet, peeling back layers, and reading source code in search of the causes of our pain. Experience and technique are the salves to this wound. </span></div><h3 style="text-align: left;"><span style="font-family: arial;">Lacking Experience</span></h3><div><span style="font-family: arial;">If you lack the experience that's ok. Just take a deep breath and approach the problem methodically. Try small experiments. Work quickly but in a disciplined manner. With each experiment, write down what you tried and what the result was. Keep trying until you are exhausted. Then, if you still don't have it, seek expert advice. Maybe the problem is intractable and that's ok too. Just back up and look for another way to solve the bigger problem. </span></div><h3 style="text-align: left;"><span style="font-family: arial;">Leverage Comes from the Fulcrum</span></h3><p><span style="font-family: arial;">If you do solve a core problem you won't just have fixed one thing, you will have fixed many things. You've fixed that core problem, but you have also fixed all the symptoms of the core problem. If the core problem was really deep in the system the impact will be huge! </span></p><h3 style="text-align: left;"><span style="font-family: arial;">Whack A Mole</span></h3><p><span style="font-family: arial;">So what happens if we don't fix the core problems? </span></p><p><span style="font-family: arial;">You will spend a lot of time and money playing Whack A Mole. Every time you fix a problem that isn't the core problem you are likely to create or reveal another problem. If that doesn't happen, the core problem will find a way to reveal itself again. Sometimes you will fix an issue and a few weeks later fix another related issue and not even notice. That is because you didn't do the analysis and figure out what the core problem is. </span></p><h3 style="text-align: left;"><span style="font-family: arial;">Pyrricic Victories</span></h3><p><span style="font-family: arial;">Let's go back to that heat-death thing. </span><span style="font-family: arial;">Look at what you are doing and ask yourself, is my investment in the current solution so great that fixing it is as effective as replacing it? Because sometimes fixing it creates so much damage in the system you'd have been better off leaving it alone than fixing it. </span></p><h3 style="text-align: left;"><span style="font-family: arial;">Expenses</span></h3><p><span style="font-family: arial;">While you are playing Whack A Mole you are wasting the resources of the organization. You are creating a barrier between you and value delivery. Remember that our job as software engineers is to create and deliver value. </span></p><h3 style="text-align: left;"><span style="font-family: arial;">Sunk Cost</span></h3><p><span style="font-family: arial;">Often we fall into the <a href="https://thedecisionlab.com/biases/the-sunk-cost-fallacy" target="_blank">Sunk Cost Fallacy</a> and that further prevents value delivery. Keep in the back of your mind that software is supposed to be soft. If the best-right thing is to rip out the broken thing and replace it, then do that. But do it for a reason, not emotional, but justifiable. </span></p><p><span style="font-family: arial;">Don't keep banging on the problem, core or otherwise, without considering that eliminating the cause is the right solution. And think big scope here, not what does this cost in my sprint, but what does it cost my organization?</span></p><h3 style="text-align: left;"><span style="font-family: arial;">Conclusion</span></h3><div><span style="font-family: arial;">We don't spend enough time talking about these subjects. We don't spend enough time educating everyone about the causes of strife and pain in the organization or the systems we are building. We certainly don't do a good job of teaching diagnostic techniques. </span></div><div><span style="font-family: arial;"><br /></span></div><div><span style="font-family: arial;">Taking the time to find the root cause of an issue and fixing that, rather than the myriad of symptoms caused by that problem is well worth the effort. Being open to radical solutions is also necessary for creating effective solutions. Save your duct tape for more <a href="https://www.duckbrand.com/stuck-at-prom">interesting projects</a>.</span></div>IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0tag:blogger.com,1999:blog-3519539694496702060.post-9741030813665150062023-02-24T15:08:00.001-06:002023-02-24T15:08:52.969-06:00Locking It In<p><span style="font-family: arial;">So in an odd set of coincidences over the past few days, I've engaged in several conversations about working with legacy code. One particular thing that migrated to the top of the conversation was locking in current behaviors. Somehow those conversations all degraded into a hairsplitting vocabulary conversation. So I thought I'd capture my thoughts and share them.</span></p><h3 style="text-align: left;"><span style="font-family: arial;">Pedantic-ness</span></h3><p><span style="font-family: arial;">One of my great frustrations within my professional world is the degree to which we are pedantic. Collectively we vacillate between being very precise and totally vague with our language. The convenience of vagueness is that we don't have to establish a common vernacular. The problem is that it can lead to misunderstandings and miscommunication. On the other hand, pedantic arguments often lead to frustrating if not infuriating conversations about the meanings of words. I submit that pedantic-ness is wasteful. Don't be a pedant.</span></p><h3 style="text-align: left;"><span style="font-family: arial;">My Bias</span></h3><div><span style="font-family: arial;">I consider myself expedient. That is, I'm not much for wasting too much time. So all the pedantic hairsplitting is nonsense. This isn't really fair and I understand that sometimes you gotta stop back and explain yourself. Especially when addressing topics across cultural boundaries. We could all probably do better at the listening thing and the thoughtful consideration thing. But judgments based on unswerving or blind opinions are a nuisance to be avoided.</span></div><h3 style="text-align: left;"><span style="font-family: arial;">The Hair</span></h3><div><span style="font-family: arial;">So the hair that got split this week was Pin-down Test. We were discussing <a href="https://en.wikipedia.org/wiki/Characterization_test">Characterization Testing </a>as a component of Legacy Rescue and techniques for ensuring that we don't break existing functionality when adding a new feature to a system. I'll spare you the details as much as possible here but, the term 'pin down' was used to describe why we would characterize a system. Something along the lines of 'We are adding these tests to pin down the behavior of the system'. </span></div><h3 style="text-align: left;"><span style="font-family: arial;">Gnashing of Teeth</span></h3><div><span style="font-family: arial;">Once the term pin down came out the conversation devolved into a violent agreement that pin-down testing is a thing, a good thing, and distinct from characterization testing BUT that pin-down should not be used in describing characterization testing.</span></div><h3 style="text-align: left;"><span style="font-family: arial;">Waste</span></h3><div><span style="font-family: arial;">So there was a protracted discussion of these things. Mostly it was all in agreement, that they have the same intent, to describe the current behavior of a system without a value judgment on rightness or wrongness. The conversation took a considerable amount of time and I categorize it as waste because it was more about egos and the need to be more formally correct (pedantic-ness) than it was about developing clarity. </span></div><h3 style="text-align: left;"><span style="font-family: arial;">Clarification</span></h3><div><span style="font-family: arial;">So the result of our collective dance was this. </span></div><div><span style="font-family: arial;">Pin-downs are about dissecting code; they enable refactoring and allow us to break down legacy code and they may include a bunch of scaffolding and other edifices that are (or should be) disposable. Pin-downs are often used to identify a defect in a system and then as a sentinel to identify the resolution.</span></div><div><span style="font-family: arial;">Characterization tests on the other hand are intended to be permanent additions to a test suite to ensure that there are no changes in behavior. They assume that the current behavior is the correct behavior.</span></div><h3 style="text-align: left;"><span style="font-family: arial;">Summary</span></h3><div><span style="font-family: arial;">So the specific issue, characterization v. pin-down results in a subtle clarification. In essence they both lock in behavior so we can tell if we're breaking things. The temporality of tests and their attended scaffolding is one of the distinguishing characteristics. Furthermore, the purposes of creating such tests are distinct variations of each other. If ever a hair was split so finely. </span></div><div><span style="font-family: arial;"><br /></span></div><div><span style="font-family: arial;">My point though is, much of the discussion had at the time was a waste. Debating the use of 'pin down' in the context of characterization is just flexing. As I've read and discussed this topic over the past few days it's really forced me to focus on the meanings of these terms and then be exasperated by the English language. </span></div><h4 style="text-align: left;"><span style="font-family: arial;">Personal Note:</span></h4><div><span style="font-family: arial;"><span style="background-color: white;">Thanks to Tim Ottinger @tottinge and Stephen Cavaliere @SteveCavaliere among others for the discussion and thoughts on this particular topic</span></span></div>IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0tag:blogger.com,1999:blog-3519539694496702060.post-6711678866504648122023-02-20T15:16:00.003-06:002023-02-20T15:16:17.495-06:00Where is Rich?<p><span style="font-family: arial;"> Hey all, sorry for the delay in my posts. I was on vacation and I forgot to post that I was going away for 2 weeks. I'll be back with a post this coming Friday.</span></p>IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0tag:blogger.com,1999:blog-3519539694496702060.post-74310243768260309822023-02-03T08:38:00.000-06:002023-02-03T08:38:07.845-06:00A foolish consistency...<p style="text-align: center;"><span style="font-family: arial;"><i> "A foolish consistency is the hobgoblin of little minds" -- R.W. Emerson</i></span></p><p><span style="font-family: arial;">I used to hate that quote. I hated it because people would misquote Emerson all the time by leaving off the 'A Foolish' at the beginning. Consistency has its purposes.</span></p><p><span style="font-family: arial;">I <a href="https://blog.noradltd.com/2023/01/special-flowers.html">previously wrote </a>on code being consistent in its structure in order to reduce cognitive load. That same thinking can be applied elsewhere.</span></p><h3 style="text-align: left;"><span style="font-family: arial;">Standards</span></h3><p><span style="font-family: arial;">Recently I've been working with my team to develop standards and processes for our organization. We're focused at the moment on identifying known standards and capturing those in one place. But in some cases, the known standards aren't written yet. This is a big problem in the sense that there are dozens of standards and keeping them consistent and well-organized is proving difficult. </span></p><p><span style="font-family: arial;">We are doing this because we are trying to create consistency across our organization. The consistency that we believe will enable us to easily automate processes. Our mission is to reduce the friction of putting code into production. </span></p><h3 style="text-align: left;"><span style="font-family: arial;">My Personal Dichotomy</span></h3><p><span style="font-family: arial;">Among the dozens of identified standards we have identified are choices about tools, technology stacks, and language standards. I personally am working on the standard for Python development. </span></p><p><span style="font-family: arial;">Anyone who's worked with me before should know that I'm personally indifferent to style guides in programming. That is, I don't care what the standard is, I care that the standard exists. You'll also know I'm not a big fan of <a href="https://peps.python.org/pep-0008/">PEP-8</a> or the defaults in </span><span style="font-family: courier;"><a href="https://pylint.org/">pylint</a></span><span style="font-family: arial;">. So maybe it's ironic that my standard on Python says 'start with PEP-8 and use </span><span style="font-family: courier;">pylint</span><span style="font-family: arial;">'.</span></p><h3 style="text-align: left;"><span style="font-family: arial;">Consistency Can Increase Value</span></h3><p><span style="font-family: arial;">I won't drag you through my philosophical arguments against the defaults. I want to talk about value delivery and speed and how any standard enables that. </span></p><p><span style="font-family: arial;">Having a programming standard that is enforceable (and hopefully auto-fixable) gives me an advantage when I code. Taking 30 minutes to setup </span><span style="font-family: courier;">pylint</span><span style="font-family: arial;"> and my IDE for auto-formatting lets me stop thinking about formatting and style. Those things become one less thing I need to do before creating a pull request. Minimally I don't have a colleague checking the rules for me. Ideally, my IDE corrects the issues as I create them. That allows me to work on problems that really matter.</span></p><h3 style="text-align: left;"><span style="font-family: arial;">Not Foolish, Pragmatic</span></h3><p><span style="font-family: arial;">The consistency of the standard isn't foolish. We're making this choice to remove a whole realm of discussion from the production of software. We are working to reduce friction and speed up development so we can deliver more value more often. Part of that is the standard itself and how it impacts our reasoning about code and part of that is leveraging or enabling automation. Remember, it's a choice to do things consistently, the standard is mostly an opinion on correctness. Don't blindly follow a standard, understand why you have it and what you get for having it. My advice is 'steal a standard' and tweak it, but don't '<i>not have a standard</i>' and find a way to automatically enact and enforce the standard.</span></p><p><br /></p>IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0tag:blogger.com,1999:blog-3519539694496702060.post-54638842033058230282023-01-26T17:21:00.003-06:002023-01-26T17:21:39.272-06:00Auth0 - Testing APIs with Live Users<p><span style="font-family: arial;">So I've been working on a API for a few weeks now. Big issues with security in my testing is that I was cheating, testing behind the authentication/authorization tier. That's fine for some tests, but we need to prove that the system is secure. In particular, we're using Row Level Security and we need an undeniable way to ensure that the holder of the Bearer token is allowed to see the data they are requesting. I'll spare you the gory details of how our RLS works and the nuances of getting a user ID associated with particular data sets (thought thats interesting too). The issue we ran into was, how do I get the Bearer Token from a Python Script using username and</span><span style="font-family: arial;"> password. </span></p><p><span style="font-family: arial;">This seems like it should be a trivial thing, but it was not. In fact, there were many hoops jumped, misunderstandings, and retrospectively trivial discoveries that need to be made. So in that light, I'm memorializing that journey in this post. Again, I will spare you the details and myself the indignity of describing all the missteps in the hopes that this post can be used as a direct guide to solving a problem for others (and a cheat sheet for me in 6 months when I've forgotten how to do this).</span></p><h3 style="text-align: left;"><span style="font-family: arial;">Setup Auth0:</span></h3><p><span style="font-family: arial;">So obviously you need an Auth0 account setup with an application or API configured and at least one user configured and authorized appropriately. </span></p><h3 style="text-align: left;"><span style="font-family: arial;">Setup Python:</span></h3><p><span style="font-family: arial;">In our case we're working in Python 3.7 using <a href="https://docs.pytest.org/en/7.2.x/" target="_blank">PyTest</a> and the Auth0-python library. The first thing we need is some fixtures to sign us in and get a bearer token.</span></p><h3 style="text-align: left;"><span style="font-family: arial;">Environmental Considerations</span></h3><div><span style="font-family: arial;">In the code examples below all the UPPER_CASE stuff are either constants or environment variables. I used </span><span style="font-family: courier;">dotenv</span><span style="font-family: arial;"> to configure the environment and excluded the </span><span style="font-family: courier;">.env</span><span style="font-family: arial;"> file from git. I won't expose those details here for obvious reasons. </span></div><h3 style="text-align: left;"><span style="font-family: arial;">Create some PyTest Fixtures</span></h3><p><span style="font-family: arial;">Since we only want to sign in once, we'll use a few session level fixtures. </span></p><p><span style="font-family: arial;">First we need to initialize the API to get a Token, this is pretty trivial;</span></p><pre style="background-color: #2b2b2b; color: #a9b7c6; font-family: "JetBrains Mono", monospace; font-size: 9.8pt;"><span style="color: #bbb529;">@pytest.fixture</span>(<span style="color: #aa4926;">scope</span>=<span style="color: #6a8759;">'session'</span>)<br /><span style="color: #cc7832;">def </span><span style="color: #ffc66d;">auth0_get_token</span>():<br /> <span style="color: #cc7832;">return </span>GetToken(AUTH0_DOMAIN)<br /></pre><p><span style="font-family: arial;">Second we need to get an active Auth0 client connected to our management API</span></p><pre style="background-color: #2b2b2b; color: #a9b7c6; font-family: "JetBrains Mono", monospace; font-size: 9.8pt;"><span style="color: #bbb529;">@pytest.fixture</span>(<span style="color: #aa4926;">scope</span>=<span style="color: #6a8759;">'session'</span>)<br /><span style="color: #cc7832;">def </span><span style="color: #ffc66d;">admin_access_token</span>(auth0_get_token):<br /> mgmt_api_url = <span style="color: #6a8759;">'https://{}/api/v2/'</span>.format(AUTH0_DOMAIN)<br /> token = auth0_get_token.client_credentials(MGMT_API_CLIENT_ID<span style="color: #cc7832;">, </span>MGMT_API_CLIENT_SECRET<span style="color: #cc7832;">, </span>mgmt_api_url)<br /> <span style="color: #cc7832;">return </span>token[<span style="color: #6a8759;">'access_token'</span>]</pre><p><span style="font-family: arial;">And lastly we need to get the bearer token itself</span></p><pre style="background-color: #2b2b2b; color: #a9b7c6; font-family: "JetBrains Mono", monospace; font-size: 9.8pt;"><span style="color: #bbb529;">@pytest.fixture</span>(<span style="color: #aa4926;">scope</span>=<span style="color: #6a8759;">'session'</span>)<br /><span style="color: #cc7832;">def </span><span style="color: #ffc66d;">bearer_token</span>(auth0_get_token):<br /> <span style="color: #cc7832;">return </span>auth0_get_token.login(WEBAPP_CLIENT_ID<span style="color: #cc7832;">,<br /></span><span style="color: #cc7832;"> </span>WEBAPP_CLIENT_SECRET<span style="color: #cc7832;">,<br /></span><span style="color: #cc7832;"> </span>getenv(<span style="color: #6a8759;">'TEST_USER'</span>)<span style="color: #cc7832;">,<br /></span><span style="color: #cc7832;"> </span>getenv(<span style="color: #6a8759;">'TEST_PASSWORD'</span>)<span style="color: #cc7832;">,<br /></span><span style="color: #cc7832;"> </span><span style="color: #aa4926;">scope</span>=<span style="color: #6a8759;">'openid profile email'</span><span style="color: #cc7832;">,<br /></span><span style="color: #cc7832;"> </span><span style="color: #aa4926;">realm</span>=<span style="color: #6a8759;">'email'</span><span style="color: #cc7832;">,<br /></span><span style="color: #cc7832;"> </span><span style="color: #aa4926;">audience</span>=AUDIENCE<span style="color: #cc7832;">,<br /></span><span style="color: #cc7832;"> </span><span style="color: #aa4926;">grant_type</span>=<span style="color: #6a8759;">'password'</span>)[<span style="color: #6a8759;">'access_token'</span>]</pre><h4 style="text-align: left;"><span style="font-family: arial;">Use the Bearer Token in a Test</span></h4><div style="text-align: left;"><span style="font-family: arial;">In the end, that was all pretty trivial. So, now we can build a test using that </span><span style="font-family: courier;">bearer_token</span><span style="font-family: arial;"> fixture like this;</span></div><pre style="background-color: #2b2b2b; color: #a9b7c6; font-family: "JetBrains Mono", monospace; font-size: 9.8pt;"><span style="color: #cc7832;">def </span><span style="color: #ffc66d;">test_using_the_wrong_customer_id_fails</span>(bearer_token):<br /> customer_id = <span style="color: #6897bb;">999<br /></span><span style="color: #6897bb;"> </span>test_user = getenv(<span style="color: #6a8759;">'TEST_USER'</span>)<br /> event[<span style="color: #6a8759;">'headers'</span>][X_CUSTOMER_ID_HEADER] = customer_id<br /> event[<span style="color: #6a8759;">'headers'</span>][<span style="color: #6a8759;">'Authorization'</span>] = <span style="color: #6a8759;">'Bearer {}'</span>.format(bearer_token)<br /> expected_response = <span style="color: #6a8759;">'cannot find user {} at customer {}'</span>.format(test_user<span style="color: #cc7832;">, </span>customer_id)<br /><br /> resp = lambda_handler(event<span style="color: #cc7832;">, None</span>)<br /><br /> <span style="color: #cc7832;">assert </span>resp == {<br /> <span style="color: #6a8759;">'body'</span>: json.dumps({<span style="color: #6a8759;">'error'</span>: expected_response})<span style="color: #cc7832;">,<br /></span><span style="color: #cc7832;"> </span><span style="color: #6a8759;">'headers'</span>: DEFAULT_HEADERS<span style="color: #cc7832;">,<br /></span><span style="color: #cc7832;"> </span><span style="color: #6a8759;">'statusCode'</span>: <span style="color: #6897bb;">400</span><span style="color: #cc7832;">,<br /></span><span style="color: #cc7832;"> </span>}<br /><br /><br /></pre><p><br /></p>IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0tag:blogger.com,1999:blog-3519539694496702060.post-17035517492884140732023-01-20T00:21:00.001-06:002023-01-20T00:21:12.131-06:00Special Flowers<p><span style="font-family: arial;">Wow, it's been a really long time. I got distracted. Writing, for me, is a challenge. Talking on the other hand :-D So, I'm back, sorta. I'm going to try to get back into the blog cycle.</span></p><h2 style="text-align: left;"><span style="font-family: arial;">Special Flowers</span></h2><p style="text-align: left;"><span style="font-family: arial;"><i>(or Speaking of Design Consistency)</i></span></p><p><span style="font-family: arial;">So what do I mean about consistency? The shape, feel, and structure of the code is more or less the same. Some people call this coherent design. Others refer to micro-patterns. I kinda like the term uniform design. Whatever word you want to use the point is that what you see in one part of the code is more or less the same as anywhere else in the code base. </span></p><h3 style="text-align: left;"><span style="font-family: arial;">Here is an example. </span></h3><p><span style="font-family: arial;">You may have made a choice to throw exceptions in order to communicate issues with validation. So you might have a function in your application that reviews the content of an HTTP request looking to see that the necessary parts are present. If something is missing you throw an exception. </span></p><p><span style="font-family: arial;"><script src="https://gist.github.com/rdammkoehler/43fd463df5b4ca573695a9a35e6e4896.js"></script></span></p><p><span style="font-family: arial;">My expectation after seeing that approach is that it will be used everywhere in the code base. Every time we validate something we would expect an exception to be thrown if the validation fails. That would be 'uniform'.</span></p><p><span style="font-family: arial;">OK, so what happens in your head when the next validation code you examine returns true if the content is valid and false if it isn't. Fair enough approach, but not the same as the first one. Not uniform.</span></p><p><span style="font-family: arial;"><script src="https://gist.github.com/rdammkoehler/4db72edff048dc0d73999cfa9145e7de.js"></script></span></p><h3 style="text-align: left;"><span style="font-family: arial;">Why does it make a difference?</span></h3><p><span style="font-family: arial;">I can give you at least two other ways that validation could be implemented. But two is one too many already. This inconsistency causes me to stop and think. What is the author doing? Is there a special reason that we're using true/false here and exceptions elsewhere? Now I have to research that and understand both. If I'm really diligent, I'll hunt down all the validation patterns and determine how many variations there are, try to figure out which is the predominant approach, and at least make any new validator consistent with that pattern. Very time-consuming. </span></p><h3 style="text-align: left;"><span style="font-family: arial;">What's Our Job Again?</span></h3><p><span style="font-family: arial;">Our job as Software Engineers is to deliver value to the business. We measure that in terms of the features delivered. So thinking about how we can deliver features faster is pretty essential to our jobs. There are 1000 ways to deliver faster and we can get into some of those in another post. For this topic, I want to point out, that making yourself work harder is not an effective way to increase the rate of value delivery. </span></p><h3 style="text-align: left;"><span style="font-family: arial;">Flowers: I only like tulips</span></h3><p><span style="font-family: arial;">So there should be no special flowers in the code. Ideally no variance in design and approach. If we can achieve that goal we reduce the total cognitive load necessary to understand the code. This is akin to pattern recognition (Sparrow's Deck anyone?). We should feel like, with respect to a particular code base, I've seen one validation block, I've seen them all. At least in terms of the mechanics. </span></p><h3 style="text-align: left;"><span style="font-family: arial;">Benefits</span></h3><p><span style="font-family: arial;">By establishing a uniform design we are reducing the amount of work we need to do to gain comprehension of the code. That means we can learn the whole code base faster. If we can learn the code base faster we can make effective changes sooner. If we make changes sooner we can deliver value to the business quickly. That is our job* </span></p><p><span style="font-family: arial;">Related, and probably fodder for other posts, by reducing the difficulty of comprehending the code we increase the available time for carefully considering innovations. Imagine not feeling rushed to 'clean up' the code because you know the pattern and so you built it 'to spec' the first time; or at least close enough that it is not an arduous task to align with the uniform design.</span></p><h3 style="text-align: left;"><span style="font-family: arial;">What's Next</span></h3><p><span style="font-family: arial;">Looking back on my post history I've really taken plenty of time off. I'm going to do better in 2023. It's not a resolution, it's just what I will do. I'm open to writing based on prompts so if you have suggestions hit me up on Twitter <a href="https://twitter.com/net_zer0" target="_blank">@net_zer0</a> I have in mind a lot of posts around code quality, still more on TDD, and an update on my thoughts around Architecture and the role of the Architect in an organization.</span></p><p><span style="font-family: arial;">* There is a lot to our job. Features are (potentially) value. Value delivery is how we get measured. I'll write more about the other aspect of our jobs soon. </span></p>IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0tag:blogger.com,1999:blog-3519539694496702060.post-33690699369100062472022-02-10T13:53:00.004-06:002022-02-10T13:53:45.177-06:00Is anyone paying attention?<p> <span style="font-family: arial;">Is anyone out there still paying attention to my blog. I know I haven't written much in several years. I've been busy. But I have a lot of ideas brewing for things I'd like to write about. Is it worth it to write blog articles? Would I be better off doing videos on YouTube or Twitch? What do my dear readers think?</span></p><p><span style="font-family: arial;">Also, I'm open to topic suggestions.</span></p><p><br /></p>IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0tag:blogger.com,1999:blog-3519539694496702060.post-46647752839768772742020-09-10T09:59:00.001-05:002020-09-10T09:59:10.629-05:00Abject Failure<p> So I keep trying to write and failing. I'm crazy busy these days it seems. I will try to do better, there is something rattling around in my head about testing, TDD, etc.</p>IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0tag:blogger.com,1999:blog-3519539694496702060.post-25831339973744255362020-07-21T09:48:00.003-05:002020-07-21T09:48:55.392-05:00It's been a while...So my life has been a bit more than crazy the past few years and I've been away too long. I'm going to start posting again. I'm keeping the bar lowish at 1 post per week and we'll see how it goes. This is not that post.IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0tag:blogger.com,1999:blog-3519539694496702060.post-21978329897240437252018-09-08T11:55:00.000-05:002018-09-08T11:55:19.885-05:00TDD, Declarative Code, Deleting Tests, an Important Nuance<span style="font-family: Arial, Helvetica, sans-serif;">I've had something stuck in my craw for about a year now. Some debates and discussions of TDD led me to this condition that I was stuck for an articulation on. I think I got it now and I want to share it.</span><br />
<h3>
<span style="font-family: Arial, Helvetica, sans-serif;">The Core of TDD</span></h3>
<span style="font-family: Arial, Helvetica, sans-serif;">TDD is about driving out a solution from tests. That is to say, rule 1, tests are the cause of code in production. The goal is to drive out a solution. It should be minimalist, it should be clean, and it should only do what the test say it should. This is well understood.</span><br />
<h3>
<span style="font-family: Arial, Helvetica, sans-serif;">Deleting Tests</span></h3>
<span style="font-family: Arial, Helvetica, sans-serif;">One topic that seems to come up a lot is deleting tests. When do you do it? Why do you do it? </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">As I've said before, I generally don't delete tests unless they are wrong. That isn't to say that I never do, but I don't spend much time worrying about it. I've built plenty of large systems with thousands of test and never had a problem caused by too many tests. It just isn't a thing. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">What I am likely to do is replace a test. That is, if I see a particularly bad example of a test -- I might kill it off and replace it with one or more cleaner, more concise tests. </span><br />
<h3>
<span style="font-family: Arial, Helvetica, sans-serif;">Communicating Intent Through Tests</span></h3>
<span style="font-family: Arial, Helvetica, sans-serif;">Tests have a secondary function. Communication. When I want to understand the authors intent, I can look at the tests and understand what the author was driving at; in a well tested system at least. I don't always have to find the author, or track down the story card, or have a protracted and speculative conversation about what the author intended. I can read the tests and understand what is supposed to be happening. Admittedly I might lack the Why, but What is half the battle.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">So, when we talk about deleting tests, I feel like that is an affront to a secondary characteristic of TDD. It offends me. We are deleting the expression of What the code under tests should do; how it behaves.</span><br />
<h3>
<span style="font-family: Arial, Helvetica, sans-serif;">Declarative Code and Tests</span></h3>
<span style="font-family: Arial, Helvetica, sans-serif;">Anyway, this leads me to the bit that has been bugging me. Deleting tests of declarative code. Or tests for 'obvious code'. The argument that '<i>The code explains itself and the tests don't add value</i>' does <b>not</b> resonate with me. Follow along here, see if this make sense to you.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><i>If the tests are the What</i></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><i>And the code only satisfies that What</i></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><i>And I delete code, I can recover the What by looking at the test.</i></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;"><i>If the tests are the What</i></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><i>And I delete the tests</i></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><i>I don't know What is supposed to be happening.</i></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">It is hubris to presume that we are smart enough to know or infer what the authors intent was. Further, it is wasteful -- why speculate when we can know?</span><br />
<h3>
<span style="font-family: Arial, Helvetica, sans-serif;">An Example Case</span></h3>
<span style="font-family: Arial, Helvetica, sans-serif;">The common occurrance of this debate topic lately seems to be around declarative code. Take this example;</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">class RuleApplier {</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"> private static final Map<Class, List<Rule>> ruleMap = new HashMap<>();</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"> static {</span><br />
<span style="font-family: Courier New, Courier, monospace;"> /* code that loads up all the various possible rules goes here */</span><br />
<span style="font-family: Courier New, Courier, monospace;"> ruleMap.put(BlueThingy.class, Arrays.asList(new BlueThingRule1(), new GenericThingRule()));</span><br />
<span style="font-family: Courier New, Courier, monospace;"> }</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"> public static final void applyRules(Thingy thingy) {</span><br />
<span style="font-family: Courier New, Courier, monospace;"> for (Rule rule : ruleMap.get(thingy.getKey())) {</span><br />
<span style="font-family: Courier New, Courier, monospace;"> thingy.apply(rule);</span><br />
<span style="font-family: Courier New, Courier, monospace;"> }</span><br />
<span style="font-family: Courier New, Courier, monospace;"> }</span><br />
<span style="font-family: Courier New, Courier, monospace;">}</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Should there be a test that indicates that ruleMap should contain a rule for BlueThingy? Should it be specific about what rules are applied? </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I'd say yes!</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">You howl, Why? Thats silly? Thats testing structure and implementation? </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I submit the following defense.</span><br />
<h3>
<span style="font-family: Arial, Helvetica, sans-serif;">In Defense of Not Deleting</span></h3>
<span style="font-family: Arial, Helvetica, sans-serif;">If I test drove the solution I would have started with a test that says something like;</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">public class RuleApplierTest {</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"> @Test</span><br />
<span style="font-family: Courier New, Courier, monospace;"> public void blueThingyGetsBlueThingRuleAndGenericThingRuleApplied() {</span><br />
<span style="font-family: Courier New, Courier, monospace;"> BlueThingy thingy = mock(BlueThingy.class);</span><br />
<span style="font-family: Courier New, Courier, monospace;"> when(thingy.getKey()).thenCallRealMethod();</span><br />
<span style="font-family: Courier New, Courier, monospace;"> InOrder ruleApplications = inOrder(thingy);</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"> RuleApplier.applyRules(thingy);</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"> ruleApplications.verify(thingy).apply(isA(BlueThingRule1.class));</span><br />
<span style="font-family: Courier New, Courier, monospace;"> ruleApplications.verify(thingy).apply(isA(GenericThingRule.class));</span><br />
<span style="font-family: Courier New, Courier, monospace;"> }</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">}</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">There is nothing about that test that says RuleApplier has a declarative block of code, or even that it needs one? It only says RuleApplier will apply those two rules when given a BlueThingy.</span><br />
<h3>
<span style="font-family: Arial, Helvetica, sans-serif;">The Nuance</span></h3>
<span style="font-family: Arial, Helvetica, sans-serif;">If I delete this test '<i>because it is testing declarative code</i>' I'm removing the what. So later, when I wonder about why, I won't have the benefit of a what to clue me in. In fact, I can now easily break the system. And surely you agree, relying on an integrated test is a bad choice; integrated tests are a sham.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">There, I finally got that out of my system.</span>IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0tag:blogger.com,1999:blog-3519539694496702060.post-60394574307434328232018-06-24T22:28:00.000-05:002018-06-24T22:28:56.565-05:00Flow Control with Exceptions<span style="font-family: "arial" , "helvetica" , sans-serif;">Recently the topic of using exceptions for flow control reared its ugly head. This topic seems to show up in my life every few years so I thought I'd share some things I've learned over the past 25 years of dealing with exceptions.</span><br />
<h3>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Don't Do It!</b></span></h3>
<span style="font-family: "arial" , "helvetica" , sans-serif;">OK, first, just don't. Don't use Exceptions explicitly for flow control. In fact, don't use Exceptions if you can help it. Exceptions should be the result of something essentially beyond your control. The name says it all, Exceptions are exceptional -- your handling of an exception should be to deal with the unexpected, despite how cynical you might be.</span><br />
<h3>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>General Handling of...</b></span></h3>
<span style="font-family: "arial" , "helvetica" , sans-serif;">So you really should try to avoid handling exceptions. That is, you should only handle an exception you can do something about. A typical good pattern for any piece of software is to have one exception handler at the top (closest to the invocation point) and handle everything there, usually with a polite message indicating that a system error has occurred.</span><br />
<h3>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>I Can Handle It</b></span></h3>
<span style="font-family: "arial" , "helvetica" , sans-serif;">There are some exceptions you can handle. File Not Found Exception is a pretty common one that you can generally handle. Now by handle, what do I mean? Well, in some cases it might mean printing a helpful error message for the user. In other cases I might mean creating or downloading the missing file, or using a default configuration. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">When you are doing this, you are not using an Exception for flow control. You are using an exception to identify and handle an unexpected (but possible) condition in your application. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Some other pointers for handling exceptions include, handle them immediately and concisely. That is, don't try to over generalize the handling of possible exceptions (other than the aforementioned top level handler). When exceptions are handled, get to the point, handle them quickly and without too many gyrations, then resume normal flow. </span><br />
<h3>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><b>Where Does It Get Messy?</b></span></h3>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Things usually get messy in highly modularized code bases. For example, if you have 20 libraries as dependencies to your application, but you wrote all of those libraries and your application, all of this is your code. This can make it hard to understand when you are using an exception for flow control and when you are dealing with things outside of your control.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">An easy way to work through this is to consider what you'd do if the library throwing the exception was an open source library, what would you do then? Would you still throw the exception? If you wouldn't do this to a stranger on the internet, don't do it to yourself.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Similarly, if the library throwing the exception was some OSS library you'd pulled off Maven Central how would you handle the exception? Same rule applies to the library you wrote.</span><br />
<h3>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Don't Over Complicate</span></h3>
<span style="font-family: "arial" , "helvetica" , sans-serif;">As with most things, it is best if we don't overcomplicate the matter at hand. Exceptions are part of our languages. There are penalties to using them, but there are also advantages. When considering how to use an exception, think about the developer who comes after you. What will make sense to them? That is what you should do. When in doubt, ask someone how they would expect things to work. </span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0tag:blogger.com,1999:blog-3519539694496702060.post-51295808204791006452018-06-19T16:09:00.000-05:002018-06-19T16:09:57.115-05:00Automate Everything<span style="font-family: "arial" , "helvetica" , sans-serif;">Stop me if you have heard this one before. No don't read this again.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Back in 1988-89 I had a job as an assistant systems operator working for a really cool guy named <a href="https://www.linkedin.com/in/jasongutenschwager/" target="_blank">Jason</a>. Mostly I ran backups and did other really simple SysOp work and I probably spent more time learning csh and making patch cables for the machine room than doing much else. But I still learned a lot in this job. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">The most important lesson I learned was, automate everything.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">It came up one day that there seemed to be a lot of idle time in the life of a SysOp. Roughly 80% of the time was available for projects like 'make patch cables' or 'clean the attic'. So I asked Jason one day, </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">"How is it that we have so much spare time? When are we doing to do some SysOp-ing.?"</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">He said, "We are! Everything is automated. When I come into the office in the morning I check my email. I review the reports generated by the automated scripts, and if nothing is wrong I have to make stuff up for us to do all day." </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">At the time it was sort of a "Ha Ha" moment and I didn't think about it too much. Years later I realized, Jason and the other Real SysOps™ had automated every single tasks they had to perform on a regular basis. They needed guys like me to change the tapes in the ExoByte drive, but not much else. And as long as things went well, there wasn't much to do.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">That left lots of free time for other pursuits. Like thinking about how to make things better, more automated. They were basically working to eliminate their own jobs. As a consequence they could work on more interesting things (homework, pet projects, etc.). I wish I'd had a clue back then, but I have one now. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">By automating away all the mundane things we can create more space to think through tough problems, innovate, or just generally <a href="https://daedtech.com/dont-learn-to-code-learn-to-automate/" target="_blank">sleep better</a>. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">So have been applying this sort of thinking since back in the day, generally with good success. I admit, sometimes it takes me a long time to figure out how to automate things. I certainly have grown to despise things that are hard to manipulate with scripts and macros. What I've gotten in the end is a fairly simple life. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">One example is a side project I'm working on. I've automated nearly everything. I did it in the Unix Way (small, atomic/acidic scripts that only do one thing). I can use all that automation to my advantage. When my partners in mischief call with an issue I can usually bang out two or three simple commands to 'fix things'. Or send instructions like "Run script X. Delete thing Y. Then restart with command such-and-such". Honestly, if I could anticipate the contortions in advance, I could get most of this down to one script.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">What this has given me is the opportunity to think about the Hard Parts™ of the system and then arrive at clever solutions. Rather than spend days trying to build a DAL for the application, I spent a day deriving a generic library that works across all of the domain objects and tables. How'd I do that? Well, I didn't spend all day manually coding up a bunch of one-off objects, I automated the construction, testing, and deployment of those things. The test cycle is about 6 seconds. I was able to iterate over my clever solution so fast that it was almost (but not quite) painless to create. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Automation is your friend. It may not be sexy and glorious, but it will enable you to do great things. So go out there and automate everything.</span>IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0tag:blogger.com,1999:blog-3519539694496702060.post-30052286524587014112018-06-18T10:00:00.000-05:002018-06-18T10:00:00.520-05:00Clever is the Enemy of Good, Part send(f(time.now)+hostname)<span style="font-family: "arial" , "helvetica" , sans-serif;">So in a recent coding adventure I came across some really super things. One of my favorites worked as follows. </span><span style="font-family: "courier new" , "courier" , monospace;"></snark></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">* Get a reference to a production domain class that contains list of event types</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">* Get the names of the event types as strings</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">* Split the strings on '.'</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">* Use the last element of the returned list to create a snake case string (from the camel case value)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">* Use send to find a method on the current object with the same name as the string</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">* Assemble the results into a list</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Now I'm down for some good old fashioned reflection/introspection and general meta-programming. There are plenty of times where its the right thing and it makes sense.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Your test setup code is not this place. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">This example I've laid out took about 20 lines of setup code and resulted in roughly this;</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">let(:event1) { Event1.new }</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">let(:event2) { Event2.new }</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">let(:event3) { Event3.new }</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">let(:events) { [ event1, event2, event3 ] }</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Why would you put all this complicated junk in your test? </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I have only one guess, Future Proofing. The only genuine motivation I can see for using a complicated setup for such a simple thing is a presumption that one day there will be more events and we want to test them all. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">This is wrong thinking. First, don't future proof your test code. It will be necessarily vague and not result in anything very helpful or useful in the future (that might not come). Second, you've now made a simple thing very complicated to the detriment of readability. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Our first goal in TDD is to understand our system; to determine what code must be created by explaining it in terms of test code. Something like this is clearly not the development of understanding. I'm pretty confident that its an example of test after development, although I didn't check. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">One of the secondary effects of TDD is that we leave behind an explanation of how the system works. Not of how it was implemented necessarily, but of what we expect it to do. Having a let() that is 20 lines long and uses reflection to assemble a list of 3 items is not clear, concise, or helpful. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">So in both cases a test like this misses the mark for good TDD. </span>IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0tag:blogger.com,1999:blog-3519539694496702060.post-30974544363495730322018-06-15T10:00:00.000-05:002018-06-15T10:00:03.184-05:00TDD Preconditions, Moar Design Pressure<span style="font-family: "arial" , "helvetica" , sans-serif;">As test drivers we need to listen to that design pressure and simplify.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I recently spent several days dissecting a a single RSpec file that was 1300+ lines long. My pair partner and I extracted a single context of 250 lines into a new file and hauled 105 lines of setup code along for the ride. There were 103 let statements and two subjects. Thats not to mention the event machine testing mix-in and the various event mothers. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">In the end we got it working but it took far longer than it should have. There was plenty of time spent questioning our understanding of the system and how it should actually behave. Had we extracted the correct setup? and did this test work before we did the extract? became our repeated refrain. Therefore we were constantly flipping back and forth with another branch and running the test suite to ensure that we weren't screwing things up. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We got the job done, but here are some things we learned.</span><br />
<div>
<br /></div>
<div>
<span style="font-family: arial, helvetica, sans-serif;">1) Tests with preconditions aren't really helpful in explaining anything to the reader. It seems like the should be, but the just kept confusing us. In fact, once we became familiar with the test configuration (getting the file trimmed down to < 300 lines) they were redundant. This is clear evidence that, if the test module is properly formed, the preconditions aren't necessary; hence they are a smell.</span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Have you ever seen a test that looks like this;</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<script src="https://gist.github.com/rdammkoehler/62054547bf54ae5567539c78010c6fcb.js"></script>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">I don't like this test. The precondition (the assertion before the execution) is telling me something is wrong. Mostly what it is telling me is that the system is complicated enough that I need to establish the current state before I can even start executing. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">Thats a design smell if there ever was one.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span><span style="font-family: "arial" , "helvetica" , sans-serif;">What that precondition is telling me is our test has become so complicated we are unsure of how the setup works and therefore our test code needs a test. Thats bad. </span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">2) (off topic but important) Reasonable defaults to you aren't necessarily reasonable to anyone else. When you are dealing with 1000 lines of test code and numerous external factories and fixtures you can get lost and confused very quickly. It doesn't help if an external testing library sets up conditions that aren't explicit but have significant consequences. Clever is the enemy of good. Don't use an unusual setting or configuration just for fun in your defaults, and if you do, make it super obvious that you are doing so or the developer who comes after you might spend a day chasing their tail.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">3) Most importantly. Listen to the design pressure your tests provide. If you feel compelled to make an assertion about the state of the system before you execute the code under test, your code is telling you 'Hey, I'm complicated!'. Part of our goal is to not have complicated things. So do something about it. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<br />IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com1tag:blogger.com,1999:blog-3519539694496702060.post-32273325949880783932018-06-14T08:59:00.005-05:002018-06-14T08:59:52.606-05:00Sekuretee!<span style="font-family: Arial, Helvetica, sans-serif;">Good morning internets. Just wanted to point out that this blog is now available via https, so update those bookmarks to https://blog.noradltd.com </span>IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0tag:blogger.com,1999:blog-3519539694496702060.post-88274600300992271922018-06-13T10:52:00.000-05:002018-06-13T10:52:38.903-05:00Listen to Setup Pressure<span style="font-family: "arial" , "helvetica" , sans-serif;">Joining an existing project can be an overwhelming experience. It seems most often when I start a new engagement there are a half dozen technologies that I haven't got installed,or have the wrong version of, or maybe haven't ever used. There are configuration files, manual setup steps, and other churn to work through just so you can run a build, let alone do any meaningful work at all.In many cases this configuration period can take an entire week and it is often very difficult to tell if you have done it correctly.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">My best ever project in this regard took 2 hours to configure. We spent over a month building an automated script to assemble everything. On joining the project the instructions were these;</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">* Download and install Java (I think it was 1.2)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">* Download and install ANT</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">* Connect to source control and pull the project</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">* Run ant build_all</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">* wait 90 minutes</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">* done.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Our build script was pretty smart. It would download, install, and configure your IDE, Database, Application Server, test suite, supporting tools, compile, package, and build everything, and run the full test suite. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I wish that every project could be like this.It paid off tremendously. Every team member who joined went through this process, every time we replaced a laptop, we went through it again. Every time we saved days of work we would have done manually otherwise.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">On top of that, everything was (for some definition of it) documented by the ANT script and the files that supported it. Everything inherently had configuration management via the ANT script. And, it wasn't even that complicated to follow along with what it was doing. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We religiously maintained that script through the entire project.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Almost (but maybe not quite) every project I have ever been on has required me to kill at least a day doing setup work. There are passwords to exchange, keys to setup, IP restrictions to modify, let alone all those tools, packages, and settings to adjust. Make it worse, if you work with more than one client you might have to figure out how to get multiple configurations to cooperate. Its a tragic mess. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I recently started with a client where I spent the better part of four days configuring my laptop. I recklessly trashed other configurations on my system to ensure that I was setup for them; so I spent no time trying to make things work in two worlds. Several weeks later I **think** my environment and configuration are correct, but I'm not really sure. Nobody is really sure. To make matters less comfortable, the configuration is touchy. Because it isn't fully automated, if I delete the docker volume for the testing database, there are six or seven manual steps that I need to execute to restore it. I had to put those into Evernote because they aren't documented well and they aren't particularly obvious (to me). </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">The team doing this work are a great bunch of people and they mean well, but they have a lot of pressure to deliver and have not had the time to automate these parts of their configuration. They are overburdened with new work, defects, and operational maintenance, and other tasks and have not found the time to go back and clean these things up. That's despite their very strong desire to do so. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I suspect that they, as a group, have become numb to the time consuming activities of fixing their configurations and setting things up; likely they are only working on this one thing and don't get clobbered by having multiple projects on their systems. For them, it's become a distant memory of a growing pain from years back. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">This configuration friction is more than a growing pain. It is a design pressure and it's trying to tell us something. Mostly, your system is complicated. That complexity needs to be dealt with. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">We can talk about the monetary cost of all this, but that is somewhat academic. What is more interesting is the psychological impact this has on a team. It plays out one of two ways, both bad. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">A team might avoid updates/upgrades/changes, or relegate that work to 'system experts' in order to avoid the distant but remembered pain of dealing with the complicated setup, OR the team will become more and more lax about how the configuration works and is documented; basically they will ignore the issues in the hopes that they go away. Both of these are costly in terms of time and money, and risky in terms of stability and sustainability. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">If all the work is pushed off to 'system experts' there are huge risks. One, that the team doesn't really understand its own systems configuration, and two, that the experts will fly-away with the information leaving the team an archaeology project when change is necessary. Furthermore, it constrains the breadth of decision making to a select few, possibly risking opportunities for innovation and growth. Both not good.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">More obviously, if the team simply ignores the problem there is a high potential for the system to become stale and prone to security exploits. Its important to keep systems up to date with respect to the technology stack both from a security perspective and an operational cost perspective. If the libraries get too far out of date, or the OS requirements get sun-set-ed, what will you do? Spend a month doing an upgrade? Do you trust your test automation that much?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Lastly, and maybe most importantly, working with such systems can be demoralizing. Starting up on a project where you feel 'dumb' for a few days or weeks while you try to work out the configuration isn't a lot of fun. Worse, when you start to feel inhibited by the difficulties of configuration, like you can't change things because of the risk of breaking things, you may start to loose your drive to innovate and make the best possible solutions.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Setup and build automation are serious aspects of good software development and should be treated as such.It may not be the sexy and glorious activity that you want to engage in daily, but it supports those things, and when done correctly gives you more freedom to do whats most fun about your project.</span><br />
<br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0tag:blogger.com,1999:blog-3519539694496702060.post-69635802146042119872018-05-29T09:38:00.003-05:002018-05-29T09:38:47.891-05:00Continuous Integration, its about time<span style="font-family: Arial, Helvetica, sans-serif;">My feed has been full of strange stories and interesting posts lately. This morning I found this <a href="https://apiumhub.com/tech-blog-barcelona/true-continuous-integration/" target="_blank">conversation about Continuous Integration</a> and it got me thinking about that practice. Specifically it got me thinking about that I was taught and how it has changed in practice. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Fifteen years ago I stumbled upon the practice of build automation entirely by accident. I was running a small team of distributed developers and I didn't want to wait six weeks for them to get their development environments working. I wrote a massive Ant file that assembled all the tooling, configuration, and resources necessary to start development on day one. It took 90min to run the </span><span style="font-family: Courier New, Courier, monospace;">setup-all</span><span style="font-family: Arial, Helvetica, sans-serif;"> target. When it was done you'd have Ant, Eclipse, DB2, Oracle, WebSphere, all the code, everything built, and any-all tests executed. I event went so far as to build a server that, given and ant build script would run it for you (presuming it could get ahold of and configure the resources). I didn't know what I was doing, I'd never heard of cruise control, CI, or any of that. I was just solving a problem. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">As inelegant as my solution was, it worked. I was optimizing the initialization of work phase of my project. In retrospect, it was because I had no idea what we were going to do. </span><span style="font-family: Arial, Helvetica, sans-serif;">However, I was looking at the south side of the north bound donkey. </span><span style="font-family: Arial, Helvetica, sans-serif;">In typical fashion, once we started work, each of us had our 'area' of the application and we mostly didn't step on each other. Along the edges I managed to stay involved</span><span style="font-family: Arial, Helvetica, sans-serif;"> and smooth over the conflicts. Some pretty sever </span><a href="https://en.wikipedia.org/wiki/Big_Design_Up_Front" style="font-family: Arial, Helvetica, sans-serif;" target="_blank">BDUF</a><span style="font-family: Arial, Helvetica, sans-serif;"> happened to minimize issues. In the end it all worked out.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Fast forward about a year, I got introduced to the world of Extreme Programming. I'd seen some books (I think I read the forward to XP Explained and discarded it by this point) and I learned about this strange practice called continuous integration. It made total sense. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">The way I heard it was, integrate everything as often as possible and test it to make sure it works before you do anything else. This lead me to development practices like pulling code 3x a day and integrating to what I was working on...no matter what. So every morning I'd synch with source control (CVS and Subversion back then) and resolve all my conflicts. Then run everything and make sure it still worked. I'd do the same after lunch, and once more before I knocked off. I wasn't even paying attention to the CI server -- I couldn't even tell you now what server we had, though I suspect it was either Hudson or Cruise Control. In any case, I did this pretty consistently and you know what, it didn't slow me down a bit. In fact, I'm pretty sure it saved me a lot of pain in the long run.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I read the aforementioned article this morning and was reminded of this. In that article there is an example of a merge conflict (kudos to author at <a href="https://apiumhub.com/" target="_blank">Apimhub</a> for taking on that topic, painful to describe cleanly) and propose at least one strategy of dealing with the issue -- a meeting. Back in the early oughts that would have been my answer too. However I don't think I've had one of those kinds of discussions in nearly a decade. Here's why.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">First, you should be rebasing or merging any branch, if you use such a thing, very frequently. Depending on the rate of change in your repo, 2-3x a day. If you are doing that, you'll likely see the intersection well before you've gone down the rabbit hole into <a href="https://www.solutionsiq.com/agile-glossary/integration-hell/" target="_blank">integration hell</a>.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Second, the integrator takes responsibility for the integration. So, whomever is doing their local merge or rebase takes responsibility for making it all work out. That doesn't mean don't ask for help, just that the only one hung up should be the receiver of a pull from master. I know this sounds cruel, but if you have 10 developers banging away on different parts of the system you can't reasonably stop all of them when there is an issue. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">This last bit has some interesting consequences. One is, it should cause everyone to make smallish commits and smallish changes. That might not happen until someone gets bit by the mother of all refactors (which coincidentally isn't a smallish thing) but once it does people will adapt to this mode <a href="https://en.wikipedia.org/wiki/PDQ" target="_blank">PDQ</a>. It will, likely, change the modularization of the system (for the better). In order to avoid the conflicts developers will start isolating their components from the other components of the system so that the integration point is very clearly defined and as small as possible. This will minimize friction when trying to integrate. (I recognize that there can be some negatives too, but that will need to wait for another discussion).</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">The third thing that prevents the BDUF conversations and coordination is community. We should not develop in isolation if at all possible. That is, we should be in more or less constant contact with the other developers on our team. That is what Slack, Gmail, etc. is for. Or if we're all in the same room, that rarely used voice thing we have. Every day we have standup, everyday we talk about what each of us has and will do. We should all have some general sense of where those things intersect. If we don't, we should step back and think about that too (maybe another discussion here?). Also, generally, we sort of know in most cases who's turf is who's. Joe is the 'server guy', Terri is the 'UI dev', and Sara is omnipresent. If we have concerns about the intersection of what we are doing we should proactively discuss them with the team. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Anyway, back to the general Continuous Integration discussion.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Overall Continuous Integration is about time. Its very much about saving you time, but its also about where you spend your time. The way I heard it back in the day was, rather then spend 6mo at the end of a project integrating everything, spread that time out across the project time line, doing a little integration at once and you will, overall, reduce the total time integrating AND you (probably) won't fail to integrate. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">As I've described it, maybe you think CI will increase the amount of time you spend doing development. It might. But you aren't the only one we are worried about. We are worried about the overall viability of the system in production and it sustainability within the organization. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">A </span><span style="font-family: Arial, Helvetica, sans-serif;">person/pair</span><span style="font-family: Arial, Helvetica, sans-serif;"> might spend say 10% more time working on any given task as a result of the repeated rebase/merge step, but if that translates into nearly zero tail-end effort to merge to master and a coherent design in the end it was well worth it. Waiting to do the integration at the end of a long branch lifecycle will just move the integration time to one place and likely cause others to get drawn into the time. So one person/pairs 10% becomes two </span><span style="font-family: Arial, Helvetica, sans-serif;">person/pairs</span><span style="font-family: Arial, Helvetica, sans-serif;"> 10%. Worse, if, at the time of integration a significant issue arrises with the design, in the name of expedience and getting things done, some design tragedy might occur, leaving us with code that works, but is ugly, hard to understand, etc. That could lead to maintenance concerns later that are costly. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">So, consider that Continuous Integration isn't about Servers, or even about building artifacts or build automation even. Continuous Integration is about saving you time in the long run, its about coherent design, and product viability. CI Servers, artifacts, etc. are all great things, but that isn't what the practice is really about. Our goal as always is to develop the best solution to a problem in a manner that is sustainable by the organization. CI is just one small but important part of reaching that goal.</span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;">I'll save feature toggles and other integration strategies for another discussion. I hope this post helps you in some way.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<br />IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0tag:blogger.com,1999:blog-3519539694496702060.post-28267444454689294452018-03-16T14:37:00.000-05:002018-03-16T14:38:16.865-05:00Why TDD?<span style="font-family: "arial" , "helvetica" , sans-serif;">Two days ago I read the article </span><a href="https://builttoadapt.io/why-tdd-489fdcdda05e?_lrsc=c6115042-875f-46bb-ae19-b0bfe46b5f1a" style="font-family: Arial, Helvetica, sans-serif;" target="_blank">Why TDD</a><span style="font-family: "arial" , "helvetica" , sans-serif;"> by </span><a href="https://twitter.com/moonmaster9000" style="font-family: Arial, Helvetica, sans-serif;" target="_blank">Matthew Kane Parker</a><span style="font-family: "arial" , "helvetica" , sans-serif;"> and I really enjoyed both the article and accompanying</span><span style="font-family: "arial" , "helvetica" , sans-serif;"> video. That said, I have a small disagreement with the content that Matt asked me to clarify after my </span><a href="https://twitter.com/net_zer0/status/974278912313364480" style="font-family: Arial, Helvetica, sans-serif;" target="_blank">tweet</a><span style="font-family: "arial" , "helvetica" , sans-serif;">.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Matthew puts forth the assertion that our goal is to 'Go Fast, Forever'. I'm not so sure about that. The goal, in my mind, is to delivery value; to make or save money as an over simplification. I'm probably picking nits here, but going fast is part of the iron-triangle under the goal of value delivery and forever implies that there is no end to our task. Practically speaking there must be an end to our task, and it is one that we can realistically predict; when the cost of development/maintenance exceeds some percentage of the future return on investment, we stop.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">So, how does this relate to TDD. I completely agree that TDD enables speed and helps assure quality and correctness, however, <a href="http://blog.noradltd.com/2017/03/the-essence-of-tdd.html" target="_blank">The Essence of TDD</a> is to attain an understanding of what needs to be done, the rest of it is a happy coincidence. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">So, if our goal is to deliver value, we need to understand how that value manifests itself, then what to build that delivers on that. We need to understand what needs to be done to achieve that goal, and we do that through TDD (and other things). </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">So consider that tests on the outside of the system prove that the manifestation of value is present, while tests interior to or specifically about the system express the understanding of the 'what', ideally in terms of behavior, that delivers the value. TDD enables the development of the understanding and coincidentally provides other benefits like regression testability, design pressure, safe refactoring, etc. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Don't get me wrong, the later all all great things, and TDD is likely the very best way to get them; but it isn't the only way. TDD, however, is the most direct way to develop a understanding of the system incrementally. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">ps - Matthew has written a number of articles that I would recommend you check out, specifically <a href="https://builttoadapt.io/four-goals-of-a-good-test-suite-651b3a4cfd33" target="_blank">Four Goals of a Good Test Suite</a>. </span><br />
<br />IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0tag:blogger.com,1999:blog-3519539694496702060.post-54024090324483140992018-03-07T22:21:00.000-06:002018-03-07T22:21:02.090-06:00A One Liner to Collect Your Python DepsSo I had a need to collect the name, version, and license info for all of my applications dependancies. I sought out and tried a few open/free tools but could not get what I needed from them (many just didn't work).<br />
<br />
I crafted this bash-one-liner to solve my problem;<br />
<br />
<script src="https://gist.github.com/rdammkoehler/1a0ebbcac95c14729bd5761f217ecb55.js"></script>
The results look like this;<br />
<br />
<pre>adal 0.5.0 MIT
appnope 0.1.0 BSD
args 0.1.0 BSD
asn1crypto 0.24.0 MIT
astroid 1.6.1 LGPL
attrs 17.2.0 MIT
</pre>
<br />
I hope this can help someone else too.IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0tag:blogger.com,1999:blog-3519539694496702060.post-31555380895721405012018-03-07T22:12:00.000-06:002018-03-07T22:12:02.909-06:00Mob Programming Anti-Pattern?<span style="font-family: "arial" , "helvetica" , sans-serif;">One of the teams I'm working on is using mob programming. I think I've identified an anti-pattern here. Mobbing can devolve into something akin to Brook's Surgical Team. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Sometimes, you end up with one mobster who's driving everything. It's as if nobody else is required. The only way you know it's still a mob is that this mobster isn't typing, they are telling someone else what to type. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: arial, helvetica, sans-serif;">I'm not talking about someone who just provides direction for a short while, I'm talking about 'all day'. What's worse is, at the end of the day it seems like nobody complains. </span><br />
<span style="font-family: arial, helvetica, sans-serif;"><br /></span>
<span style="font-family: arial, helvetica, sans-serif;">Mobbing takes some discipline. You have to push back when someone acts like this; speak up in retro. If they won't change their behavior, leave the mob. Eventually they will just be working on their own.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0tag:blogger.com,1999:blog-3519539694496702060.post-46418629973496103732017-11-15T09:33:00.002-06:002017-11-15T09:33:41.461-06:00Architects in an Agile/Lean World<span style="font-family: Arial, Helvetica, sans-serif;">I just read this very <a href="https://martinfowler.com/articles/ea-in-lean-enterprise.html" target="_blank">nice post</a> from Martin Fowler on <a href="https://martinfowler.com/articles/ea-in-lean-enterprise.html" target="_blank">The Role of an Enterprise Architect in a Lean Enterprise</a>. He has a lot of good thoughts in there. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I submit that there is one aspect lacking emphasis in this post however, the responsibility of the agile team to interact with the architect. That is, agile or not, architecture is still a thing in every organization. At some time, some person made a variety of choices about how systems should be structured, interconnected, and in some cases to be implemented. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Those decisions were not made lightly or in a vacuum and often the consequences of changing them have far reaching consequences that may not be apparent to those without knowledge of the entire enterprise system. Consider that an architects job among other things is to deliver value to the business over an extended period of time. Part soothsayer, part Oracle, an architect should be working to further enable business objectives through technology, not just in the near term, but for the foreseeable future of the organization. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Architects, like everyone else, are fallible human beings and they cannot foresee every circumstance. Any given project may run afoul of gaps in a generalized architecture or may struggle to integrate two technologies due to any number of compatibility factors. These are times when an agile/lean team should be reaching out to the architect for help. As Martin points out, part of the architects job is not to say 'No' but to make sure the team is fully aware of the consequences of deviations from the current technology stack. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Failure to consult with the architect when these decision points are reached is just plain negligence. Furthermore, failure of the architect to respond and cooperate with the team is a gross dereliction of duty. Like everything else in the agile/lean world, the relationship is a partnership with a shared goal and it is incumbent on all parties to participate in that partnership; to cooperate to find solutions.</span>IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0tag:blogger.com,1999:blog-3519539694496702060.post-2005669860845975342017-11-06T08:33:00.003-06:002017-11-06T08:33:36.241-06:00Vague<span style="font-family: Arial, Helvetica, sans-serif;">I spent most of my weekend working on my NaNoWriMo book. If you aren't familiar with NaNoWriMo, the idea is to write 50,000+ words in a month, basically a novel. The degree to which you prepare for this event is entirely up to you. I've been talking about what I would write about since July, but I haven't really developed the details. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">So, I'm trying to write about gang members in Mexico, but I don't have a town name, a general description of the town, etc. I haven't even named the gang. Its all pretty vague.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">So the problem with that is, nothing in the descriptions is named. Or if it is, it's in general terms. The Pontiac. The Boulevard. The Restaurant. Obviously, those are details I'll need to go back and fill in at some point. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I've witnessed a parallel in project teams. Teams where an idea is discussed, changing the messaging infrastructure for example. The idea is discussed periodically in non-specific terms over an extended period of time, but there are never any details. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Two things tend to happen. Often, because there is nothing concrete, the work never happens; it remains a goal that is unachieved. The other is, the work is attempted, but it is a horrid mess of a change because there wasn't agreement about the 'how and the what'. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Don't get me wrong, I'm all for late binding on the 'how and what', but that ultimately has to happen. There should be a progression of work toward that 'how and what', maybe a series of spikes to prove or disprove things about the intended approach, but those need to be focused a little. Someone, someplace, has to provide a specific objective or set of criteria in order to focus the work.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">There are a number of ways to get there. The Architect, as a technical product owner, can specify a set of behaviors required of the system. The Tech Lead could specify an approach, either in how the interaction should work, or even some specifics about the tool. The Director of Engineering might also have requirements about cost, platform selection, etc. </span><span style="font-family: Arial, Helvetica, sans-serif;">Without these hints and nudges a team could drift aimlessly trying to make the change happen. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">One last thing that is essential. The work has to get scheduled. It won't just happen in between other cards on the board. The product management team must make a change important and schedule it; set a goal date for completion, or otherwise do something to cause the work to happen. Even if a development team agrees upon the importance of a change, they are beholden to the planned work. If it isn't on the schedule, it just wont happen.</span>IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0tag:blogger.com,1999:blog-3519539694496702060.post-61142353055059541482017-11-03T10:50:00.000-05:002017-11-03T10:50:22.906-05:00Microcosm <span style="font-family: "arial" , "helvetica" , sans-serif;">I live in a microcosm. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">So last week I went to SCNA and talked to a bunch of people there that I don't talk to enough. I've also been having lots of conversations about How it Should Be Done™<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">So many of the people I've been working with over the past 10 years or so have very similar back grounds, have read the same books and white papers, and worked in the same environments. There is a significant degree of group-think involved. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Recently though, I've been exposed to some new people who don't have those characteristics, or rather, come from a different group. I'm learning a lot about how other people think about problems and I'm learning a lot about how other people think. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I don't agree with some of the things I'm learning; which is to say, I've learned a few new techniques and ways of looking at problems and I don't agree that they are more effective or better than what I was already doing. I'm trying to be careful to avoid a false confirmation on these things of course.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">What I find most useful about this experience is that I gives me a change to question how I've gone about solving problems for the last 25+ years. Some days it sucks, because you stand there questioning something you've been doing for 10 years and wondering if you've been 'wrong' for that long. Sometimes it's awesome because you get validation of your technique.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">All of us live to some degree in a bubble created by those who surround us. All of us could be fooling ourselves every day in terms of </span><span style="font-family: "arial" , "helvetica" , sans-serif;">How it Should Be Done</span>™<span style="font-family: "arial" , "helvetica" , sans-serif;"> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I highly recommend that you get out, meet some new people (or read some new blogs) and consider that what you think you know as the right thing might be improved upon.</span></span>IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0tag:blogger.com,1999:blog-3519539694496702060.post-52676395341339873702017-11-01T08:31:00.000-05:002017-11-01T08:31:01.694-05:00NaNoWriMo<span style="font-family: Arial, Helvetica, sans-serif;">Today is the first day of National Novel Writing Month. I'm going to be one very busy camper. I hope to be able to get a few posts out each week, but coverage may be spotty. Stick with me though.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Thanks,</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">.rich</span>IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0tag:blogger.com,1999:blog-3519539694496702060.post-66760775966795716692017-10-23T08:31:00.004-05:002017-10-23T08:31:53.749-05:00Functions as First Class Citizens<span style="font-family: Arial, Helvetica, sans-serif;">So I've been doing a lot of thinking and reading on the notion of functions as a first class citizens. In Python, functions are objects, just like instances of classes are objects. So when do you create a class and when do you create a function? </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">In my mind its somewhat a matter of style. That is, there is no mandate to use a function in place of a whole class or vise versa. That said, there are certainly times when a class is way too much work and a function will do just fine.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">So a function doesn't have state, or at least it shouldn't. That means anything it knows about the world it gets as an explicit or implicit argument. Explicit arguments come in the form of parameters just like you would expect. Clearly implicit arguments should be constants or retrieved values and should never be global variables and the like.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Once you have written your function you can feel free to import it, call it, pass it around. It is pretty handy to not have to instantiate something in order to get some functionality. BUT, is this all necessary?</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Again, I think we get back to a style question. I grew up on C/C++/Java with a little Perl thrown in. Discounting structured programming, in my mind everything should be an object and every object should have a class. That is clearly a habit born of Java. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I would not be critical of an application full of objects. I would not criticize a developer for never having a stand alone function. That kind of style question does not register in my lizard brain as a concern. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">As I continue to play with the idea of functions as objects and functions standing alone I find myself questioning a lot of what I've learned to believe about good design. The biggest issue that occurs to me is, functions just laying about feels disorderly. I have to remind myself that the Python concept of 'module' as a means of organization has some significance. Think module == classy and it feels a tiny bit better. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">Anyway, there is certainly some utility advantage to having free standing functions. For one, no tedious object instantiation. Another benefit in my mind is, you don't have to prefix a function with </span><span style="font-family: Courier New, Courier, monospace;">self</span><span style="font-family: Arial, Helvetica, sans-serif;">. That is one of the least pleasant things in Python in my mind, all this </span><span style="font-family: Courier New, Courier, monospace;">self</span><span style="font-family: Arial, Helvetica, sans-serif;"> prefixing (same complaint about OO-Perl etc.). So if typing has somehow become the hinderence to your success, using standalone functions will save you some of that. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">One other thing, in the past we've had this issue of 'where do I put utility functions'. Like </span><span style="font-family: Courier New, Courier, monospace;">convert_to_utc(datetime)</span><span style="font-family: Arial, Helvetica, sans-serif;">, a function that given a datetime in some timezone will convert it to the UTC timezone. That doesn't need a utility class, we don't need/want to do </span><span style="font-family: Courier New, Courier, monospace;">Utils().convert_to_utc(datetime)</span><span style="font-family: Arial, Helvetica, sans-serif;"> and so the function cleans that up, making it just </span><span style="font-family: Courier New, Courier, monospace;">convert_to_utc(datetime)</span><span style="font-family: Arial, Helvetica, sans-serif;">. That is one aspect of the function as a standalone/first-class object that I do appreciate. No more 'common' or 'util' classes cluttering things up.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-family: Arial, Helvetica, sans-serif;">I have some other thoughts I'll share in future posts but for now, chew on the idea that functions can be first-class citizens in your program.</span>IndividualRichhttp://www.blogger.com/profile/01018677010619145900noreply@blogger.com0