Testing Python

Introduction 1 CHAPTER 1 A History of Testing 5 You Do Test, Don’t You? 7 Fundamentals and Best Practices 7 Python Installation 8 Linux 8 Mac 8 Windows 8 Pip 9 Virtualenv 9 Source Control (SVN, Git) 10 Interactive Development Environment (IDE) 11 Summary 12 CHAPTER 2 Writing Unit Tests 15 What Is Unit Testing? 15 What Should You Test? 17 Writing Your First Unit Test 17 Checking Values with the assertEquals Method 18 Checking Exception Handling with assertRaises 20 Following the PEP-8 Standard 22 Unit Test Structure 23 Additional Unit Test Examples 24 Getting Clever with assertRaises 24 Making Your Life Easier with setUp 25 Useful Methods in Unit Testing 27 assertEqual(x, y, msg=None) 27 assertAlmostEqual(x, y, places=None, msg=None, delta=None) 27 assertRaises(exception, method, arguments, msg=None) 28 assertDictContainsSubset(expected, actual, msg=None) 28 assertDictEqual(d1, d2, msg=None) 28 assertTrue(expr, msg=None) 28 assertFalse(expr, msg=None) 29 assertGreater(a, b, msg=None) 29 assertGreaterEqual(a, b, msg=None) 29 assertIn(member, container, msg=None) 30 assertIs(expr1, expr2) 30 assertIsInstance(obj, class, msg=None) 30 assertNotIsInstance(obj, class, msg=None) 30 assertIsNone(obj, msg=None) 30 assertIsNot(expr1, expr2, msg=None) 31 assertIsNotNone(obj, msg=None) 31 assertLess(a, b, msg=None) 31 assertLessEqual(a, b, msg=None) 31 assertItemsEqual(a, b, msg=None) 31 assertRaises(excClass, callableObj, *args, **kwargs, msg=None) 32 Summary 32 CHAPTER 3 Utilizing Unit Test Tools 33 Using Python’s Nose 33 Installing Nose 34 Using Nose’s Best Features 35 Running Specifi c Test Files 35 Getting More Detail with Verbose 35 Debugging Support with PDB 36 Checking Your Coverage 38 Coloring your tests with Rednose 39 PyTest: An Alternative Test Runner 40 Installing PyTest 40 PyTest’s Best Features 41 Running Specifi c Tests 41 Viewing Detail with Verbose and Summary 42 Debugging with PDB 43 Checking Your Coverage with PyTest 45 Choosing Between Nose and PyTest 46 Mock and Patch Tricky Situations 46 Installing the Mock Library 47 Mocking a Class and Method Response 47 When Mock Won’t Do, Patch! 50 The Requests Library 50 Patch in Action 50 Advanced Mocking 52 Summary 53 CHAPTER 4 Writing Testable Documentation 55 Writing Your First Doctest 56 Th e Python Shell 56 Adding Doctests to a Method 57 Running Your Doctests 58 Handling Error Cases 59 Advanced Doctest Usage 61 Improving Doctests with Nose Integration 62 Summary 65 Resources 65 CHAPTER 5 Driving Your Development with Tests 67 Agile Development 67 Adopting the Agile Process Now 68 Ethos of Test Driven Development 70 Advantages of Test Driven Development 72 Ping-Pong Programming 72 Test Driving Your Problem 73 Writing Your Failing Test 74 Making Your Test Pass 75 Driving More Features with Tests 75 Wrapping Up the Task 77 Summary 82 Resources 83 CHAPTER 6 Writing Acceptance Tests 85 What Is Acceptance Testing? 85 Anatomy of an Acceptance Test 87 Using Gherkin Syntax 87 Th e Magic Is in the Step File 88 Goals of Acceptance Testing 89 Implementing Developer and QA Collaboration 90 Letting Behavior Drive Your Problem 90 Writing Your Failing Acceptance Test 90 Defining Your Steps 92 Implementing Your Code 94 Developing More of the Feature 95 bank_apppy 96 indexhtml 97 Delivering the Finished Article 98 Advanced Acceptance Test Techniques 102 Scenario Outline 102 Tables of Data in Scenarios 103 Summary 104 Resources 105 CHAPTER 7 Utilizing Acceptance Test Tools 107 Cucumber: The Acceptance Test Standard 107 Lettuce in Detail 108 Tagging 108 Fail Fast 112 Nosetest Integration 114 Robot: An Alternative Test Framework 115 Installing Robot 116 Writing a Test Case 116 Implementing Keywords 117 Running Robot Tests 119 Summary 123 Resources 123 CHAPTER 8 Maximizing Your Code’s Performance 125 Understanding the Importance of Performance Testing 126 JMeter and Python 126 Installation 127 Configuring Your Test Plans 128 Utilizing Your Test Plans Effectively 135 Code Profiling with cProfile 135 Run a cProfile Session 136 Analyzing the cProfile Output 142 Summary 144 Resources 144 CHAPTER 9 Looking After Your Lint 145 Coming to Grips with Pylint 146 Installing Pylint 146 Using Pylint 146 Understanding the Pylint Report 149 The Module Block 149 The Messages by Category Section 149 The Messages Section 150 The Code Evaluation Score 150 The Raw Metrics Section 150 The Statistics by Type Section 150 Customizing Pylint’s Output 150 Telling Pylint to Ignore Errors 153 Covering All Your Code with Unit Tests 154 Installing Coverage 155 Using Coverage 155 Advanced Coverage Options 157 Producing an HTML/XML Report 157 Setting a Minimum Coverage Threshold 159 Restricting Coverage to a Specific Package 159 Ignoring Coverage 160 Summary 161 Resources 162 CHAPTER 10 Automating Your Processes 163 Build Paver Tasks 164 Installing Paver 164 Creating a Paver Task 164 Executing Paver Tasks 165 Defi ning a Default Build 166 Setting Up Automated Builds 168 Installing Jenkins 169 Adding Coverage and PyLint Reports 175 Generating a PyLint Report 175 Generating a Coverage Report 176 Making Your Build Status Highly Visible 176 Summary 181 Resources 181 CHAPTER 11 Deploying Your Application 183 Deploying Your Application to Production 184 Creating a Deployable Artifact 185 Defining the Paver Tasks 185 Incorporating Packaging into the Build 187 Enabling Archiving on Jenkins 188 QA Environment 189 Implementing Stage and Production Environments 190 Implementing a Cloud Deployment 191 Creating a Heroku Account 192 Creating a Small Application 193 Setting up Git for Heroku 193 Deploying the Application to Heroku 194 Smoke Testing a Deployed Application 195 Example Application Stack 196 Smoke Test Scenarios 197 Implementing Smoke Tests 198 Summary 200 Resources 201 CHAPTER 12 The Future of Testing Python 203 Stub the Solution 203 Making Deployment Natural 205 Automating (Nearly) Everything 206 Working in Public 207 Collaborating on Step Definitions 208 Final Thoughts 209 Resources 210 Index 211