DjangoCon US 2017 – Functional Programming in an Imperative World. Maybe by Derik Pell

(gentle music) – Hi everybody thanks, I appreciate it So, my name is Derik Pell

I'm an engineer at Emma Email Marketing That's me on GitHub and Twitter Actually if you go out to my GitHub, if you go out to that URL, to the DjangoCon 2017, this is, my entire presentation's gonna, is being done in a Jupyter Notebook, and I've uploaded it to GitHub, so if you can't see the slide you can pull it up in GitHub and follow along It won't run the code in GitHub but you can look at it all, clone it, run it, it's Python 3 Quick show of hands, how many people showed up hoping I was going to explain Monads? No, sorry

As far as I know they're still just burritos So, if you don't know what I'm talking about Google it Anyway, so what I want to do is talk a little bit about functional programming Functional programming has become sort of the hot new buzz word right, and, coming from an object-oriented language like Python, sometimes we might see functional programming as something mysterious I want to kind of break that up a little bit for you and introduce you to some of the tools that help functional programming, you know, do its job, and recognize that most of these tools exist in Python

Not always to the best They're not always the best tool to use but they're all there, and so, I just wanna sort of get everybody, just an overview, of what functional programming is right But let's, why would we use functional programming? Why would we stop talking about object-oriented programming and look at this new style of programming? Well, functional programming is very expressive As we will see through some of the code examples, you can do a lot with a very little bit of code You can just, you can shrink your code base pretty considerably, in the right places

It can be very efficient Things run really quickly when your language actively supports functional programming paradigm And we'll see some places in Python where it does that very well, some places where it doesn't quite so well It can be safer Hyphenated that on purpose, to sort of emphasize the fact that it's still not safe, like there's still problems, but it's still, it can be safer, it can overcome a lot of the problems, a lot of the bugs that we see, in day to day object-oriented programming

And, it's easier to work with concurrent and parallel programming We won't actually directly dig into that but that's really sort of the reason that functional programming has become so popular in the last several years, as we've stopped getting core processors that can go faster and faster and instead using more and more cores Being able to effectively take our code and spread that across a couple of different cores is the next step in making our applications run faster Functional programming can help lead into that But what is functional programming? It's really just the style of programming that utilizes pure functions, which are functions that don't have side-effects, and it uses those functions to transform data

So functional programming actually thinks about data in a very different way than object-oriented programming Object-oriented programming, we hide our data away and we're very particular about how we use it and whatever and who can access it and that sort of thing Functional programming is a more open data model and it's like no, like there's no need to hide your data away Let's, you know, take it and work on it in a more mathematical sense That's where we try to have pure functions, things that don't have these side-effects because they don't always make sense to have side-effects like in the middle of your code

On the IO boundaries, of course, you have to have side-effects, you have to write to a database, print to a screen, read from a database, that type of thing, but in the middle of your code there's a lot of places where we do side-effects where we don't necessarily have to, and functional programming is a way of thinking that helps you sort of reduce those, 'cause a lot of times those side-effects are places where you have bugs So what I wanna do though is start looking through some of the tools that help you get to functional programming There's like, functional programming is not, it's like saying, you know, if you were to explain object-oriented programming you might start with, okay, this is how you build an object But an object, building an object or a class is not the totality of object-oriented programming In the same way there's not one part of functional programming, it's really just a set of tools and the mindset on how you use them

So, with that, let's start looking at the tools The first one is higher order functions If you've never come across this phrase before, and a lot of this stuff you guys may have seen You just may not realize that it is sort of like hand in hand with more functional thinking But, higher order functions, just means that functions get treated like first-class citizens

They can be passed to other functions as parameters They can be returned from other functions, as values, and they can be stored in variables So, you know, they're just like any other piece of data in your application So let's take a look at a couple of examples So here, is something that's sort of real worldish, we do this kind of thing quite a bit at work, where we need to, we will make a function that knows how to talk to our database

It knows how to make the initialization to the database, or maybe it's just pulling a connection off a pool or something or other, but at the end of the day we have one function that knows how to grab a cursor to our database Then we have all these other types of functions that need that cursor So we may have inserts that will return a row ID right But that's gonna be different than using a query that's gonna return you an entire result set right So, instead of trying to make some, instead of trying to make each individual, either making each individual query like, make it's own connection to the database or make some function that connects to the database in a way that's virtual enough to like, return a row ID in this situation, but return an entire row set, result set in this other, what we do is we make a function that knows how to connect to the database, grabs the cursor, and then we make other functions that use that cursor and each individual function

All it needs to know is how to use the cursor It doesn't need to know how to connect to the database So what we do is we have something along the lines of call database, which takes a function, somewhere in there makes the cursor, and then, as its return value, it calls the function that we passed in giving it the cursor right So, one example might be where you query a database, we, the query database function takes a cursor and then it knows how to loop over the result set or grab the result set and pass it back or whatever it needs to do It just depends on your business logic

But the idea is that we're passing when we do this, we're gonna like, this next line down here, we're gonna take the results, we're gonna get them by calling directly to the call database function and passing it in the query database function right So, we can see here, call database is doing its thing to initialize the database, then it gets a cursor and it passes that cursor into whatever function we called it with, which in this case is query database So it's querying the database with our cursor, right But if we also need to do something slightly different we can do something like returning the new row value right So we can reuse this same call database function 'cause all it does is one thing, work on the cursor, and it just passes a cursor in, whenever we pass it the function it calls that function, passes in the cursor right

So that's something that, that's a place where we use quite often passing a function into another function You can also return functions So this is a little more contrived example This is probably not something that would actually happen quite a bit, but if for instance, let's say you had some data coming in somewhere and you had, it could potentially come in from several different sources Right it could come in from a source that needs to read off S3 or another source that needs read out of a SQL database

You can create these functions that do whatever they need to do Like, you have an S3 function that knows how to call out to S3 and retrieve the data Or another function that, like we just saw, that calls out to the database and retrieves data from the database But you need to dynamically choose which of those functions, based on the source coming in So this essentially becomes like, almost like a poor man's polymorphism, but it's a pretty straightforward example

What we do is we have a source that comes in, in this case S3, we ask for a new function by calling out to this give me a function Give me a function takes our source and says okay, well it's S3, I'm gonna return this complex S3 function to you right It doesn't call it, it doesn't do anything with it, it's just passing the function back to us, and then it's up to us, once we get this new function, it's up to us to actually call it with whatever we need to call it with Right, so we get this, you know, we're essentially saying here's, we're coming in from S3, what do I do with it? Give me some function that I know how to handle, that will know how to handle S3, or give me some function that will know how to handle SQL And because these are actually, what is passing back are functions, we don't actually have to even do this where we assign it to a value and then call it, you can just call it straight away this way right

So we're calling into, sorry, calling in to give me a function, passing it in our source, and whatever comes back we immediately call it just like we would any other function, 'cause that's all it is, it's just passing us back a function Now, anyone who's, like I said you may have seen this sort of stuff, especially this type of thing we use a lot when we're doing JavaScript, if you have to do that But, we actually use it a lot in Decorators Who has worked with, created, looked at Decorators? A lot of people, yeah, oh the entire room right So um, so yeah that's all a Decorator really is

You're passing in a function to a Decorator function, in this case decorated That has a nested function which we typically call wrapper And then it does whatever kind of work you need to do before calling your function, whatever kind of work you need to do after calling your function So, yeah This should be pretty straightforward to everybody who's ever used Decorators before, and that's all it is

So without higher order functions, Python couldn't have Decorators, so we have to be able to pass those functions around and so you see it's not just usable end functional programming, we use it all the time in object-oriented programming in certain languages And that's sort of the key, is like, functional programming isn't really a mystery It's really just a set of tools that you use in a slightly different way And the next set of tools I wanna look at is our immutable data, or immutable data structures Now this is something we don't use as much in Python because mutable data is sort of the bread and butter of object-oriented programming, but for immutable data it's a structure that never changes itself

So like you have a list, you don't actually add something new to the list, instead your list can't be changed (phone buzzing) Sorry about that Your list is sort of set in stone And we'll look at a little bit about how you overcome that 'cause that seems like a limitation right Like you create a list, you can't add to it, you can't delete from it, what good is it

How do you make any changes? You have to think about your list in a completely different way because whenever you make some update we're used to passing in a list, adding something to it, getting that list back With immutable data, you don't do that You pass in a list If you add something to it you get a brand new list back It's a completely different data structure

Same idea, if you take something off the list You know, you pass in the list to a function, it takes something off that list, what you get back, completely different list That's a very safe way of handling things though That means that list isn't gonna change out from under you How many times have you all done this, tried to track down a bug where, because we've passed off a list, we don't realize that whatever function we're passing it to is mutating that list, before it comes back to us

Well, with immutable data structures, you don't have to worry about that You can pass them a list, they can do whatever they want to with it, your list never changes Their list gets adjusted to a new list Now, we don't do this a lot because historically they have been very inefficient What you have to do is take that list, it's sitting in memory

The language has to then copy the entire list to a new part of memory, so that you essentially have two lists It's very slow to do But there have been new data structures called persistent data structures, that make this much more efficient If you download this I got a link to persistent data structures, the description in Wikipedia I'm not gonna dive into them but they're much more efficient at this sort of thing, so it lets you use an immutable data structure, without paying the additional cost of like all that copying over in memory

If you're not familiar with the problem with mutable data structures And probably almost all of you have seen something like this This is like, almost every language I've ever looked at has this sort of warning of this kind of thing where you create a list and then you create a second list, you point a second list to the first list, anytime you change that first list you've automatically changed that second list, right So, that's not a safe way of making sure your list doesn't change, you can't do that typically You can see here, list one and list two

They are the same thing, they're basically the same list, 'cause what we have are two pointers to the same piece of memory And this can be a problem here, in that sort of situation, but you can also create some really interesting problems if you're not thinking about it So this is just a quick function that goes through, takes a list of integers, and then sums up that list right, and gives you the results So, if you pass in the list one, two, three, the summation of that is six But if somewhere in there you accidentally start appending to your list, what you're gonna get are some really weird results right

You're in the middle of working your way through a list in a way that we do every day It's just a for loop But somehow we've appended that list in the middle of that loop, and now our list is gonna continue to grow Added this if clause in there, because if I don't catch for a list of a certain size, this will grow until I run out of memory So, that's one of things that we end up having to track down sometimes, when we work with mutable data

But if you work with immutable data, a lot of those situations disappear So right here, we're doing the same thing that we did above with the list right We're creating a tuple instead of we're creating a list We're creating a tuple, adding some items to it, and then creating another tuple pointer to the same tuple, right So we've got two pointers to the same tuple, just like we did with the list above

But now, when we update the one tuple, it does not update the second one, because in Python, whenever you try to update, whenever you try to concatenate an item into a tuple, you get back an entirely new data structure right So before we do the concatenation we look at these two, tuple one and tuple two, they are the same After we do the concatenation, they're no longer the same because Python has created a brand new data structure for us It's a safe way to do it, but it's slow, because these were not created as persistent data structures And you can do, typically if we want an immutable data structure, we go to a tuple

You can do something along the same lines with pretty much any, with list or dictionaries You can use a deep copy and instead of actually just making two pointers, you make a deep copy of the list That way when you change one it does not change the other, because you start off with two different lists But again, it's slow because it has to take that list, copy it over in memory So, those are some of the problems that we can see with immutable data, that we can hopefully solve with more mutable data structures

But, working with immutable data structures, means we have to start thinking about things slightly differently often Like we don't want to do a for loop on an immutable list because it doesn't loop quite as well It'll loop in Python In a lot of functional programming languages though they just pull the loop out of there because if you aren't mutating your list, you don't necessarily need loops You can come up with other ways of getting through everything in your list, and one of those ways is recursion

Or the big way is recursion If you've never worked with recursion, most of you have probably, I assume, seen it, but recursion is when you call a function from within itself It takes, like I said it takes the place of loops in many programming languages, and we'll see that here in just a moment, how we can use recursion instead of a loop The problem with recursion is that every time you make a call back to the function, you add a frame to the call stack, and that call stack can grow and grow and grow In languages that embrace recursion, they often do this thing called tail call optimization which lets you, more efficiently, do recursion, without adding to the call stack

But Python does not have tail call optimization Guido has, if you've ever looked at it, he's sort of famously come out against tail call optimization in Python because it really does, it hides your stack trace What tail call optimization does is it, instead of building up every stack, anytime you call back, if you do your recursive call properly, it just overwrites the existing frame on your stack so that you don't build, you just sort of keep reusing the same frame It's much more memory efficient but you do lose your call stack if something happens You know, five or six calls deep, you don't know exactly where it happened, you don't have all the context for it

So, we probably will never get tail call optimizations But, regardless, let's take a quick look at recursion So, this is a recursive version of the summation list that we did above So above we're giving it the list and looping through the list Here, what we're doing is we're giving it the list and then instead of looping through each item, we send the list back into the function itself

So, the function in this case has to take the list, but it also has to take the summation Because we are recursively calling back into itself, it's harder to keep track of the state You have to pass this, you typically end up having to pass the state back into the function, along with whatever, like well, in this case, we're passing the list in, we also are passing in the cumulative sum There are ways, depending on what algorithm you're using or getting through, there are ways to do this without having to pass the state back in But in this case we are

For recursive functions, you almost always have to have a base case Base case is the point that tells the recursive function, stop recursing, we're done, just return a value Don't call back into the function again So in this case we're done when the list is empty, when it's just an empty list We have summed up everything in the list, just return the summation

If that's not the case, what we do is, we pop the last item off the list, add it to the current sum, as the new sum, return everything back into the loop again right So, that's our sum list, our function, and here we're calling it, we're calling it with the list one, two, three We get our result of six Now, we still have the same issue that we had with the iterative approach which is, if this list gets changed while we're doing the summation it's going to, you know, mess up our recursive calls This does not fix that problem

That's a mutable data structure problem But, if you were to try to solve or, you know, try to implement this in a more functional language, this is the kind of thing you would do You would loop through it using a recursive call rather than an actual for loop Now, one of the problems with recursion inside Python is if you try to go too big, if you try to recurse too many times, you can get a recursion error Now, in a lot of languages you don't have something called a recursion error

Python has built in a limit on the number of frames you can put in a stack, which by default is a thousand You can up that if you need to but in, sooner or later you're gonna run out of memory If your list is big enough, if you recurse deep enough, you're gonna run out of memory because, you know, the language has to keep track of everything on the stack That's where you get a stack overflow So, in most other languages, if they don't limit your recursion depth, like Python does, then you'll just recur and keep going and going until you run out of memory and get a stack overflow

So, without tail call optimization, recursion is actually really dangerous to do so you don't see it a lot I've never seen it in the wild in Python And there's a really good reason, so if you don't know how deep your recursion is gonna go, come up with a different solution, you can always use an iterative solution There's no problems you can solve with recursion that you can't solve with just a loop in some capacity So, in Python we definitely tend to favor loops over recursion

So anyway, just some of the ways that recursion gets used and that's how we kind of get around some of the problems of mutable data, and I don't think I pointed that out well but essentially what you're doing is you're taking that list and sending a new version of the list back through the recursive loop each time In this case, we're popping the last item off but in other situations you, a lot of functional languages have this concept of the head and the tail of a list The head of the list is the first item, the tail is everything else A lot of recursive calls will use the head and pass the tail back along So that's sort of essentially what we're doing here

So yeah, those are, if you've looked into functional programming just sort of at a surface level, those are probably the minor players that we've gone over so far The big three, the holy trinity of functional programming, are map, filter and reduce And this is where the more functional style really shines out well in Python If you've never used any of these in Python, if you get nothing else out of this talk, I hope you can walk away with a sense of I need to check into map, filter and reduce and figure out how to use them 'cause they make your code, once you get used to how they work, they're beautiful So, a lot of times though, to really use these tools, map, filter and reduce, we a lot of times end up using lambdas

Lambdas are anonymous functions, and that really just means it's a function that doesn't, that we don't name right Instead of saying deaf, you know, call DB, we just have, well, this is the, this is how it falls out in Python You use the keyword lambda You pass it whatever variables you need, in this case X and Y And then there's a colon that separates the variables from the body

Lambdas are used for typically one-liners If your lambda gets much more complex than you can fit on one line, you're probably better off just making a named function, and going the typical route there But lambdas are really good for these inline one-offs where it doesn't even make sense to make an entire function just to add two numbers together, right You can just throw it into a lambda In this case we're assigning that lambda to a variable named sum, we call sum just like we would any other function and it sums up whatever we give it, right

You don't usually use lambdas this particular way but you do use them quite a bit in map, filter and reduce So let's start looking at those three tools Filter does exactly what you think it would do You pass in a filtering function, which we'll go over in a moment, and a collection to operate on The filter, the actual filter function, takes whatever filtering function you pass in, and applies it to every item in the list one at a time

So it takes the first item, runs it through your filtering function The filtering function needs to evaluate to either true or false So, if it evaluates to true, the filter will then take that item out of the list and add it to your new list If it evaluates to false, that item does not get added to your new list Technically, map, filter and reduce all return iterators now

In Python 2 they would return list but now they're returning iterators which is much better That way you can kind of evaluate that as you want It doesn't build your entire list at one time, it yields back one item at a time, so you can loop through it however you need to But essentially, if you take something like that list, if we look at our filter right here, we're gonna create a new list, iterator Using filter, we pass it in a lambda

It's basically just saying, looking at whether or not the value that we're looking at, is it even or odd right If it's even then we assume module 02 is equal to zero, then it gets added into the iterator If module 02 doesn't equal zero it doesn't get added to the iterator We're passing it list one as the collection that we wanna iterate over And so it's filtered out all of our, I guess you can say it's filtered out all the non, all the odd integers

It's filtered in the even integers, I don't know semantics But, any case, that's what filter does You can pass it a lambda like this or you can do something much more complex, as long as the filtering function ends up evaluating to true or false So filter knows whether or not to add that value into the new list The same kind of thing is true for map

Map takes a mapping function, and a collection to operate on, and then it applies that function to every item in the collection, one at a time It does not filter out any items It is not, you know, if you pass in a 10 item list you will get back a 10 item list Or you should get back a 10 item list So, but what it does is, it says, you know, I have this whole list of users and I wanna transform them in this particular way

You can use map, and give it whatever function that's gonna transform your list of users, and then give it the list and it will go through and do the transformation for you So, you, it's a simpler way of, you just don't have to call it all in a loop right Like, normally, if you're not using map you might, you know, just for every item in this list, call out to this function manually Map does it for you, map is actually a lot more efficient at it, it'll run a lot faster So map is, if you have one of those sort of simple situations where you just need to pass every item in a list, one at a time, to a function, map is a really good alternative to using a for loop

And, same kind of thing here, we're using list one We're calling map, we're passing it this lambda, this anonymous function, that all it does is, take whatever item I give it and multiply it by two And then put it back in the new list right So there, we've taken list one and we've multiplied it by two Every item in there by two, and get back a new list

And you'll notice, even though we did a filtering on the list one earlier, we didn't actually adjust list one at all We got a brand new list back Again, because this is happening, because these functions were built, map, filter and reduce, were built to work this way, they're much more efficient than if we tried to reproduce them ourselves So this is actually a safe, efficient way, to go through a list of items, and do something with it, rather than just sort of manually looping through it, risking the possibility of adjusting your list in line or anything like that So this is a really safe way of doing this type of thing

And the last of these is reduce Reduce is a little bit different than the other two Sorry So what reduce does, it starts off the same, it takes a reducing function, right, and it takes a collection to reduce, but the reducing function's a little bit different It doesn't just act on one item in the list it acts on two items in the list at a time

You can actually send it an optional initial value so the reducing function, instead of acting on the first and second item, it'll start off acting on the initial value that you send it and then the first item And the way it works is this Your reducing function takes these two items, does something to it, so that the output is one item of the same type that comes in Now, we're not strictly using type, you know, like integer versus string versus whatever But essentially, what is gonna happen is it will take the reducing function, the reducing function will take two values, do something with it, spit back out a new value

That new value then becomes the first parameter when it calls the reducing function again, and the next item in the list becomes the second parameter And so, let's take a quick look here And this, reduce, reduce was the one that, for whatever reason, was hardest for me to wrap my head around exactly what it's doing Math and filter were pretty straightforward for me, reduce was a little harder to overcome, so I've got a couple of examples that might help it solidify But in any case this first one is the sort of standard reducing example

We're calling reduce, on this anonymous function that just takes two variables, and adds them together So, just like we did above, when were looking at lambdas initially, we're just taking X and Y and adding them together, and returning that value What this is gonna do though is, it's gonna start with one and two, out of the list, that's gonna be the first X and Y It's gonna add them together and you're gonna get three Three is them passed back in as the X and the next item in the list, which is also three, gets passed in as Y

Those two get added together to get six Six gets passed back in as the X and the next item on the list is four, gets passed in as Y And at the end of the list you end up with your sum of 10 Now, this is the same kind of thing that we saw above Right, we saw an iterative approach and recursive approach to taking a list of numbers and adding them altogether

This is, using a reducer to do the same thing, and it has, it's so much better If this is the kind of thing you need to do you need to start looking at reduce For one thing, before, when we tried to use the recursive approach, we did, we added a thousand, we blew the sack right Or you had that recursive error Here, because of the way it's built behind the scenes, it adds the first thousand numbers, altogether, without blowing through the stack

It knows what it's doing But it also does it really fast That's the first thousand numbers That's the first 10,000 numbers That's the first 100,001 numbers

I mean it's super-speedy, it's incredibly efficient This is probably much faster than we could manually go It is much faster 'cause when we were doing the recursive approach above, I started it off and we had to wait for a moment for it to blow through the stack This is going through it, just lightning fast These things were made to do this type of work

They're very efficient on the back end You really need to start looking at map, filter and reduce as viable options for whatever kind of, like wherever you think you can put them, it's probably a good place to put them, because they really do make things so much easier And so much faster I wanna go through one more quick example There's a couple of examples, you can pull 'em down and look at them if you want

I like this example Essentially, what this is gonna do is, it's gonna go through and count, it'll sum up all of the A's in all three of these strings right So this is, imagine this is a DNA sequence, it's got three A's here, the next sequence has two and the last sequence has none The reducing function in here is taking A and X, in this case we're starting with initial value of zero, so the first time it goes through, the A is zero and the X is whatever the next item is In this particular case, X is that particular string

What reducing function does is it takes that string, does a count of the A's, adds that count back in to whatever the A value is, and kind of sums it up that way So this is the same kind of problem we were trying to solve above, more or less, anyway I forgot which one it was But in any case this is a really efficient way of like, you can give this huge list of DNA strains and it will go through and tell you all these markers in it, just by using a reduce You can also mimic map and filter inside Python using list comprehensions List comprehensions are something, I'm sure, every one of us has done, day in and day out

You can throw a lambda inside a list comprehension If the lambda's too ugly you can pull it out You can also throw a filtering function, more or less inside a list comprehension Anyway, so, these are tools that are already built in to Python that we use day in and day out, but we don't think of them as being functional programming, perhaps, but they really are At their core they do the same type of thing that we would do in functional programming

So that leads us to the question of, should we bring functional programming into our code, because it is a really different paradigm I think of course you should Functional, like I think I've shown in a couple of different places, functional programming can be less verbose It can be much more efficient, and it can help you think through problems in a way that maybe you weren't able to think through before So maybe solutions become more obvious when you're using a different set of tools

But there are some very real drawbacks, as we've also seen Working with immutable data can be slower, especially in languages like Python Again we have, there's a persistent data structure library that's in Python now that someone's building I don't know anything about it, I haven't had a chance to look at it If it turns out it works well then I'm going to start trying to use it more in my code because it's much safer

But immutable data is still slow in Python, the built-in versions Recursion can blow through the stack very quickly, as we've seen, so recursion is not a good solution in most cases And functional code looks sort of weird I have a co-worker who loves to throw functional code into our Python, and every time I hit it, even though I like looking at functional code, every time I hit it it's still a bit of a speed bump When I first started hitting it, it was a brick wall

So it like totally slowed me down and stopped me But now that I understand what's going on with it I can scan through it more easily and understand it So if you're going to bring functional programming into your code, probably something you need to discuss with your team or your management, if you're working on a team, and decide as a team, hey, is this something we really wanna approach because if you just start throwing it in there and your teammates don't know anything about it, they're gonna have quite a learning curve but you can help them learn, and that's kinda fun At least I think so So that's our speed run through functional programming in Python

How much time do I have? Okay, all right Does anyone have any questions? (applause) – [Woman] Thank you for the talk So you mentioned that, you know, in some situations you maybe get to bring functional programming into your code but I was wondering, it sounds like you are saying it can be possible to just have a bit sprinkled in here and there, is that right Do you recommend, you know, having it basically be the paradigm, like choosing one or the other, or do you think it can succeed if it's half and half or sprinkled in? – Yeah I would, definitely in Python, it's better to sprinkle it in Like Python, while it supports all of these tools and a whole slew of others, there's the functools library in Python, which is really great for doing these things efficiently

Python is, at heart, an object-oriented language and so making that full transition into a functional paradigm probably isn't really gonna work out There are a lot of good places where you can put it where you might not think you can One thing I didn't get a chance to touch on is the idea of data pipelines So one of the things you can do is compose functions which is sort of like, if you've ever done any Unix programming where you pipe output from one function call to another, or if you've done like maybe Node, working with promises, where you can then, the results of one function call into the next That's composing functions

You can build really great data pipelines doing that and you can do that in Python, and Python's really great with data science kind of things So there are a lot of places where you can use a more functional mindset but it's probably not the paradigm you're gonna run with completely inside Python You might be able to, and if you can let me know, 'cause I wanna know how that works out But my, I've never been able to do that – [Man] Hi, great talk, thank you

You mentioned emulating map with list comprehension I was wondering, other than considerations of who else is reading the code and whether or not they're familiar with functional programming, are there technical differences between the two and how should you decide which one to use – Oh, I don't know the technical differences There are, I'm just going to assume that there are I know the difference that you're gonna run up against would be efficiency and speed

My guess, and I, again this is just a guess, but my guess is now maps are probably, they're not gonna be any slower than doing a list comprehension I can almost safely say that They may be a little more efficient, a little faster, because of the functools that have been brought in Map and filter are still a core part of Python Reduce has to be brought, has to be imported

But I'm gonna go out on a limb and say it's a coin-toss probably more or less You might get a little more efficiency with one or the other but you're probably not gonna notice a huge difference If I'm wrong about that, I'm sure the internet will let me know – [Man] Has incorporating functional programming in your code change your caching strategies at all? – It's not changed our caching strategies I can see how it would, we just, we haven't gotten to that place yet

Really there's only one or two other people in my office who start implementing functional programming so we're still sort of like, kind of, like not even as a company we're building it out Like it's just something that some of us know about and like oh yeah, this is a really good place to, you know, throw a filter or throw a map in or whatever Our former architect loved functional programming and he's the one that first started putting it in and he's got weird functional stuff all over the place and that's where I sometimes hit a brick wall, looking at functional code But I can definitely see how it can change our caching strategies, especially since a lot of these like map and filter are returning iterators now, so they're lazy evaluations, we don't have to like cache this whole, huge, change of a list or whatever We can sort of slowly build those changes and use them as needed and then kind of get rid of them

– [Man] All right great, thank you very much – Thanks, thanks guys (applause)