We agreed to letting them audit the code with conditions.
1. The audit happened on our computers with someone from our team in control (me). I locked the computer when I wasn't physically there to watch what they did.
2. We removed the most sensitive part of the code and told them what it did. We kept the method signature.
3. All of the source code remained on my laptop and my laptop was never attached to their network.
4. We could tell them that we would not answer any question they asked.
5. They paid for expenses and time. It lead to a decent sized contract.
It lasted about 2 days for a medium sized Java application. They asked one or two questions I wasn't allowed to answer and took it well when I told them so.
I am not a lawyer but you can also ask for a non-compete and or a non-disclosure from their individual employees that will conduct the review. Get your lawyer on that before you do it.
It is up to you or your company to decide what to do. No one can tell you if it a good or bad decision for you. At least for me and my company, it worked out. As far as I know the auditing company never developed a competitor and was a customer of my company for a long time to come.
Doing what OP described is great: it lets their folks do the audit with no risk of you loosing "ownership". It shows you are both a good partner and value what you do.
You basically install a scanner on your machine, feed your sources into it, it hashes them line by line and sends hashes to the mothership for analysis. It then spits out the report that file X, line Y matches something in the open source package Z. At least that's how it worked a decade ago, when we had to do a pre-acquisition source code audit.
When was the last time some user of Windows verified whether Microsoft Windows contains some piece of code that Microsoft shouldn't be redistributing?
And again in 2015:
Anecdote: We have released code under the Apache 2 License (our biggest project by far is https://github.com/sheetjs/js-xlsx) and we've been roped into negotiations because some companies tried to take shortcuts by copying our code without proper attribution.
I've gone 12 rounds with IP lawyers over these theoretical violations (static vs dynamic links). But I found it odd that I could never find a single case of significant liability due to infringement. The nature of damages is unclear and the landscape of counter-parties (with an incentive to sue) is amorphous. It seemed like worst-case, a proven infringer just had to re-write the offending module and make a $10k donation to an open source foundation. I've never knowingly infringed GPL and am not advocating that anyone should; it's just as I said, I found the legal community's focus on this area out of step with their otherwise well-measured calculations of risk and reward.
That seems true with pretty much any legal area. But, it's not their job to calculate risk, it's their job to tell you what is legal. It's management's job to decide if they want to take the risk or take the legal advice.
An anecdote from 2001:
So, even if there never was a big payoff, the mere potential is a big red flag for the due diligence team (and leverage for negotiations as well). They can add indemnification clauses on contracts (they won't do that for a small company like the OP I guess).
https://sfconservancy.org/docs/2010-07-27_dj-opinion.pdf interesting read
Where do you work that attorneys generally make well-measured calculations of risk? And are you hiring?
Linking GPL code to incompatible code is against the license terms and since you can't copy/distribute the code without a license you violate copyright if you distribute that code. Nobody, not even the FSF, believes that this means that your code must be under the GPL at that point -- it's just the remedy that the GPL explicitly allows. Other remedies are usually possible from the copyright holder (usually just "stop infringing our copyright"), or imposed by a court.
IANAL, but the act of a user linking GPL code to incompatible code without distribution is not copyright infringement IFAICT. I believe the FSF also agrees with that. In fact the GPL specifically states that you may run the code for any purpose. This leaves a kind of grey area where you could write code that can link to GPL code, but that you leave the end user to do the final linking. In the extreme these are the so-called "binary blobs" in things like the Linux kernel. My understanding is that the FSF thinks that these are an infringement but that other people disagree. I don't think it's ever been tested in court.
Edit: I should point out that the reasoning for it being an infringement is that by intentionally writing code that can link to the GPL code, you are creating a derived work of the GPL code. Whether this argument holds water is anybody's guess and I would love to see it tested in court.
Full disclosure: founder is a friend and all around great person.
What would Microsoft say if a customer said "we need to see the source code for Windows, Outlook, Exchange and Office applications before we use them, just as a matter of license compliance"?
Wow, I've never heard of this before - it sounds great!
Do you advertise this on your website, or is it just buried in the terms and conditions? I was just wondering how you might get across this info without worrying potential customers with the mere mention of collapse?
Most companies will dictate that you use an escrow provider from their "preferred suppliers" list.
So what our clients dictate is probably irrelevant to your needs.
Or is it more a case of, if we're collapsing, we'll put this in place?
No-one would trust that. If you are collapsing you aren't going to care enough, or the people who do/did both care and know what to include (source, build toolchain setup, documentation) have already gone.
It is surprising, given how important it could be, how little the clients bother to check that escrow updates are happening - so even if the contract says otherwise it could often come down to this!
> Are you actively syncing to a private Git repo?
Some do that, but in my experience it is more common to provide a snapshot (a full copy of the relevant parts of your repo(s) with supporting documents), with a new snapshot uploaded with each major or minor release. Sometimes it isn't released based but instead the client dictates escrow is updated "at least once per year or X months" which to my mind shows that part of the contract was written by a legal/admin person without a lot of technical experience.
Either way, get the review done fast. As they say, time kills all deals.
So this is a sales objection. With regards to that justification, I'd want to know a) who in the business is generating it and b) what they expect to feed your answer into.
Is this just somebody who wanted to sound smart in a meeting? Then they don't need your source code; they need ~5 nice PowerPoint slides and you're done. I'd be polite but firm in this case.
Is this e.g. a risk management officer? Then OK, that is a totally reasonable ask. It's going to cost you $20k and some legal wrangling regarding non-disclosure and non-competes, but we will give up to 2 team members of yours up to one business day of access to our code in a clean room on our hardware, under supervision by a technical leader, with appropriate rules of engagement regarding questions.
Is this a business unit head sniffing for new things to build? Then I will not automatically eject out of this conversation, but my price just had another zero added to it, at least.
This is a compliance process and it’s controlled by the SEC.
All of your suggestions would make the company that I at least work for to simply walk away from table.
This has happened more than once and all of our contracts contain a clause that if the application does not get a pass from our application security team the procurement would not continue.
It’s also important to note that as far as I am aware contracts on this scale have multiple stages and we are paying for PoC/pilot programmes during which we usually perform our review.
This usually isn’t software that a pre-sale engineer would come and setup in a few days so we are paying effectively even for a demo.
Overall we audit everything from Xerox software to custom balance sheet management products.
The process is dependent on the level of trust/confidence we have in the vendor and the risk profile of the product.
But in all cases there will be some sort of review.
For vendors that would not give us code access we request permission to pentest the product the pentest would include reverse engineering in most cases and as the vast majority of products in the financial sector are in Java the source code isn’t hard to get through reversing the byte code.
Overall it depends on the risk profile of the product SaaS products tend to have a lower risk profile becuase they don’t deal with trades/contracts directly.
It’s all about managing risks.
One of the products we use is Salesforce.
Penetesting Salesforce would be a waste of time for us because Salesforce has a good application security team and is a trusted vendor.
However we did perform a review of apps/plug-ins that run on the Salesforce platform which we use but have much less confidence in.
Depending on the business context of a ML solution for a company like the OP describes, you're likely looking at $500k+, but this is pure speculation and requires more information about the customer situation (e.g. is the ML solution going to save the company money or help them sell more).
> All of your suggestions would make the company that I at least work for to simply walk away from table.
Unless the company is already a customer and is paying for the development of the software, please do walk away from the table, I'm sure they have better things to do and better customers to sell to
As for the second part I wont comment really I’m not in a place to make decisions on procurement I just brake things.
I would just say that size/volume wise as far as clearings go there isn’t a better/bigger client. And everyone is the same.
>Unless the company is already a customer and is paying for the development of the software, please do walk away from the table, I'm sure they have better things to do and better customers to sell to
This sentiment is similar to things I hear all of the time working in health care / biotech. The fact is you have no domain knowledge and not a clue as to how things work in the real world. If your company just ignores the way things are done and the requirements of the domain you're simply going to go out of business.
When the stakes are high (e.g. death/harm, massive financial losses, etc.) the business process changes accordingly.
I'm guessing this is a typo?
I guess I risk inviting you to reply "Yes." or "mostly yes" with that laundry list, because most are theoretically plausible signals and sort of statements of the same idea, but what was your thought process leading to the conclusion they're underpriced? As someone who works for a vendor this is relevant to my interests :)
Because the poster is surprised by this requirement and doesn't have standardized answers to it yet.
In poker, there is this thing called "assigning a range" to someone based on their actions. You can't see their cards, but their actions might give you signal where you could say "Hmm, playing like they have a middle pocket pair and not totally air nor a monster." If sophistication with regards to enterprise sales is a spectrum like poker hands, surprise about certain parts of the process causes me to adjust my range accordingly, and that plus experience working with software people colors my estimate of pricing sophistication.
> underpriced by a factor of 2X to 10X.
The OP never asked about price, so this is pure speculation. Why is the conclusion that this is a sales objection?
> It's going to cost you $20k and some legal wrangling regarding non-disclosure and non-competes
It sounds like they haven't even gotten through procurement yet, which means they can't even suggest charging $20k for NDA's etc.
> Is this just somebody who wanted to sound smart in a meeting?
This absolutely does happen, but it doesn't happen when the company is discussing POC plans.
> We know that they have lots of resources and are building up internal data science team. And yet it was pointed out to me that their goal might not necessarily be to outright steal our IP, but rather to cover their bases. But we are still worried they might be "inspired" by the parts they see and get their internal teams to replicate across other sites or use cases. And we don't have the resources to litigate, nor any way of knowing they do this.
> Is this a business unit head sniffing for new things to build?
These are both hypotheses based on observations, but they are not conclusions to the situation.
The OP's company needs someone who has experience in this realm (a "grey haired enterprise software sales guy") and help them come to a conclusion before they make decisions to move this forward. There's way too many variables and open questions to offer any real sound advice.
TL;DR - for situations this complex and high stakes, relying on a HN comments will not bode well.
For the record since someone asked in another comment, I have navigated a procurement process with a financial services company with $1T in AUM, working on a $50M/year software project, as an example of experience.
I work for a fortune 50 basically doing web server stuff. Right now our security team would like to run some startups code synchronously as a module in our web server.
Their code could easily cost us millions off dollars (if the outage was small). I need to make sure their sdk is free of race conditions, and has proper timeouts and throttling and has proper metrics.
If your product can interrupt billions of dollars in revenue, I'm going to need some assurances and "we're really careful" doesn't count.
Of course they are free to say "no", and we can go our separate ways.
Because we're public we can't always tell vendors how much money they've cost us so when they accidentally point their prod to Dev and cost us millions of dollars all we can do is try and get a credit on this months bill.
I've services described a pretty specific use case, but there are other more generic possibilities. It's great that you have magic data science sauce for scoring customers, but when I get sued for racial profiling "I didn't know they used race as a factor" might not save my ass in court.
The IPC cost should not be significant compared to the rest of the web server code.
Can also ask them to do the work and provide a small open source in-process shim that sets up and talks to the sandboxed process.
Please feel free to reach out to me (info in profile) if you ever want to have a chat or are looking for some fun work.
It's a question of devops- if this thing dies, or changes in negative ways, do I have the power to fix it internally? Or am I calling an outside company during business hours only to request that they please look at my problem?
That's the danger large companies face when deciding if they should depend on an external vs build it internal.
In such environments you anyway wouldn't deploy a new version from a vendor without extensive testing, so it's not something you do frequently and you can re-audit at the time. If you're paying a few million per year for a licence, then it's not an issue to spend $10k worth of manpower per upgrade to ensure that it goes properly.
If so, this implication was inaccurate.
Also to be more helpful it’s not unreasonable to ask to speak with current customers, seek proof of claims on stuff like SLAs and uptime, etc. There’s literally a million ways people do this in the real world without obtaining source code.
We recently looked at partnering with an analytics company for a credit scoring solution. The idea was to use their SaaS and have a contract in place regarding use, pricing, up-time etc (Corporates will want that). When we kicked off the procurement process internally an army of people got involved. Their was a project manager, a number of lawyers. I believe their were audit and compliance people their too. They ended up creating a project just to figure out what needed to be figured out. Here's where it get's helpful. We couldn't even share what we needed to know with the SaaS team or product owner because that could compromise the investigation. In this case, the corporate's concerns were genuine, numerous and required rigorous answers.
In another case I watched an exec try to figure out how a KYC solution provider has achieved the level of accuracy in identifying ad-hoc documents in the hope of solving the problem with internal resources.
If I were you I would not send them your code.
Rather say would not be closed to them asking questions and you answering by showing code as long as it's not the secret-sauce.
Just know that for our scoring problem we actually had to see how the decision was being made because we're subject to a ton of legislation around that.
The answer is a clear no. They can PAY YOU to make custom plots/charts/reporting or run queries if they want to understand what it does better. There is almost always a way to achieve any business goal without requiring source code.
The only case I can think of source code needing formal verification by a third party is if you're targeting drones to kill people or government jailing people. That doesn't sound like a commercial company in any event.
Microsoft, for example, gives source code access to paying enterprises and governments under the Shared Source Initiative specifically for security vetting and other auditing purposes.
OP: Consult a lawyer who specializes in these matters.
Not the parent, but I have done deals like this and I feel the need to counter your sentiment.
Sure, 3rd party security review, escrow, etc etc is normal.
However, this is a massive red flag for me:
>Their justification is: "we want to see how your algorithms made their decisions."
I mean this is straight out of an episode of "Silicon Valley". The OP's entire product value is in "how your algorithms made their decisions" and this is not something you want to expose to anyone unless perhaps they are about to acquire your company.
First time the big gorilla company liked our product and they wanted it for a core process of their business. They knew we were a small startup and they wanted to be sure we were doing things properly. So they asked us for a full audit of the code by a third party company. This external company was a big consultancy and auditing company and they run something like a 'due diligence' process on us and our code. Extremely professional and clear. They never had access to the full source code, only access to pieces they requested and 'only for their eyes' during a short period of time and always supervised by us. Since we wanted to close the deal, everybody worked hard to pass this test. Later on, we signed a source code escrow agreement.
Several years later, I was a working partner for a different company and same situation with a big Telco company. Since I had the experience I was involved in the deal, but this time this company did not want to play fair and it was clear that they wanted to copy the 'magic' of our solution. Their R&D division asked for the source code and they wanted it all the time they needed to study it by themselves. It was crystal clear from the early beginning they wanted to copy us, so we did not sign the deal. Then they tried to hire the core members of our development team for their R&D division.
So the lesson here is any company can screw you if they want and you don't protect yourself. You have to take risks when you are a small company, but your code and core team are sacred. I think most companies prefer to be nice guys rather than being a gang of bastards, but it's good to be cautious.
Find a good lawyer, or better get a good advisor on your board in your company to assess you on how to deal with this kind of negotiations. And never say 'no'; say 'yes, but we would like to do it under our conditions'.
Were there other signs they wanted to copy besides "asked for the source code and they wanted it all the time they needed to study it by themselves"? I'm curious about the red flags to look out for when negotiating such deals
The second company (Telco) was the opposite. The requirements were vague and its R&D team felt attacked because a local division of the company in UK chose us versus them. So I listed what they could see of our code and how (white room, our computers, no network, partnering with us all the time...) and I asked for a scoring method based on their requirements to know if we passed or not the test. They rejected all our conditions. They wanted the code to give us the “blessing”.
The funny thing about this Telco is the UK branch signed the deal no matter what R&D said. As I said R&D tried to hire our Team. They failed. Then they spent 2 years developing a copycat of our solution to replace us in UK. And when they finally were ready, the cost and impact of transition to their solution was too high, and the copycat solution was abandoned.
How to know if are good or bad guys? It’s hard. When both parts want to sign a deal everybody gets involved and pushes towards the goal. Also, connect to companies that did that before to tell you their story. For example, the Telco had an “extractive” reputation and people warned me.
Find a way to say "yes", which satisfies their need to hear you say "yes", but your "yes" conditions mean they need to spend money (which they won't want to do), and further conditions, even if they do, as other commenters have suggested, make the process dysfunctional.
Watch our politicians in government handle any issue. They are masters of saying "yes" and delivering "no", which makes people feel like they got "yes".
> Find a way to say "yes", which satisfies their need to hear you say "yes"
This is the best skill to train if you're going to be working with enterprise clients.
You might wonder how to do that without them noticing that you are backing out of all your commitments. You in effect force them to agree that the fixed price commitments that you make are only valid if the government does their bit of the project, and you lock down exactly what their bit is. Anything outside this clearly defined scope is a "variation" or "change".
You do it by identifying the grey areas and putting a condition on each grey area.
For example there might be a requirement in the tender that you'll deliver some report in a week, or build some software function in 3 months at a fixed price. You would have some fine print to say "if all client staff are available to contribute, if all systems are fully available for review and if all approvals are gained. 1 week project $2,000 additional hours $220 per hour". You know its going to take far more than one week, now you are going to get paid for it, but you charged only $2,000 for the initial week which was likely cheaper than the other tender submitting companies who you are competing with. You just got them to agree to pay you $220/hour, and they didn't realize that most of the project will actually be carried out under this bit of fine print because they didn't meet their commitment to their side of the project - bingo!
Your client can't disagree with the grey area conditions because its only reasonable that if they require the report in a week that they do their side of the project work and if they don't then you need to be paid for the extra time it is taking.
Your client will be optimistic about what they can do when it comes to what they need to contribute to the project. Take advantage of that optimism and get them to be paying you when it takes them longer than they expected to do their bit.
The real money is made in these additional terms because in many cases all the initial conditions of the contract become irrelevant for various reasons and thus the terms are dictated by the fine print that you defined, that the client is not paying close attention to during the tender assessment process because all they care about is ticking off their predefined requirements, which you already made sure you get a big "yes" tick on. This is how big companies make scads of money from government consulting contracts. They know to make their initial tender extremely appealing to the client and their fine print extremely lucrative.
Make sure you have a good project manager whose job is mainly to track actual against contract commitments and get signoff when moving to hourly instead of the fixed price commitments.
Read up on Bill Gates Windows license contracts that the hardware manufacturers had to sign .... genius.
Truth is all in how you tell the story.
I used to run a enterprise software business for many years and we learned quickly to always just answer “Yes”.
Specially with requirement lists clients would send us...much of that stuff didn’t make any practical sense and it was clear that who ever wrote them had no clue what they where doing...answering “Yes” got us into the door and we always ended up delivering what they actually needed at the end.
Just make sure to charge for every Yes they make you deliver :)
What does "cover their bases" mean?
As them to explain what they are trying to achieve and find other ways to assuage their concerns.
The only legitimate thing is to have something in case you fail and they have "banked" on you. There is a legit way to solve that. basically if they want that tell them they should pay for an Escrow service - that will hold your code and they would receive it if you ever cease to exist. But it's important that they would need to pay those fees (they are significant.)
That should make them back down.
It's entirely unreasonable for them to demand access to source code.
The government and large corporations apparently disagree, or Microsoft wouldn't have their Shared Source Initiative. And for any nay-sayers in the crowd, that should be all that need be known: Microsoft thinks it's okay, and they have a lot more to lose than you do.
The only legitimate thing is to have something in case you fail and they have "banked" on you.
Which might be the exact thing they're trying to avoid. If your "machine learning" algorithm amounts to a bunch of nested if statements, they'd probably rather not "bank on you" in the first place.
Microsoft made the decision to create Shared Source Initiative only after they were very successful and only after a very, VERY detailed cost vs. benefit analysis.
Furthermore, they are handsomely paid for it (e.g in order to be eligible, you need to pay for at least 10k Windows licenses as per https://www.microsoft.com/en-us/sharedsource/enterprise-sour...) and it's ultimately up to Microsoft to grant/deny access.
And since we're talking about Microsoft, in the early days they were infamous for pumping competition for technical information under the guise of due diligence and then crushing said competition by developing competing products.
The person who asked the question is clearly not at the "successful monopoly" stage as Microsoft but more in the "there are legitimate concerns someone might steal our core ip" stage.
National agencies such as FBI are not threat to MS to become competitor, unlike OP’s client.
Models are the output of the process. It's like going to graphic designer for work and them not giving you the PSD file. It's just not acceptable.
I have no idea what I'm talking about in this specific niche either... but does it normally go over well when you start a conversation that way?
>about: CEO at http://www.3scale.net
I'd venture to say there is at least some qualification to answer here. </sarcasm>
It's just frustrating that this poor startup is going to make a monumentally bad decision based on people in here who have no zero clue what they are talking about.
I also think it's reasonable to ask for a model so you can test and validate it yourself.
In addition, I'd love to help your confidence with getting used to the idea that saying no is ok. I remember being in this situation many times and feeling like if we said no we'd lose the client and maybe go out of business.
But truly, clients often ask for things that aren't very important to them and they will not mind being told that the answer is no.
The mistake you can make here is to blow up your no answer into a big deal.
The term I was given for how you should respond is "the principle of the simple explanation." People are totally ready to believe you, so just say no in the simplest way possible.
In the middle of some other response just include a line like:
"Re: source code access. No, our source code is proprietary and we don't share it."
Or even more simply "No, we don't share access to our source code."
E.g people are more likely to cut in line at the copier if you say 'can I cut in line, because I need to make a copy'. Even with an inane excuse like that (everyone else there also needs to make a copy) people are more likely to accept.
So your first example is likely the better one, even though the reason given is basically void of explanatory content.
For some it is more important, and they won't yield, but some proportion will doing something because the cost of doing it is low enough that there's no reason not to.
In that case giving a reason creates a cost in persisting in that you need to consciously think through whether or not it's worth arguing over, and if so what to respond. And sometimes that is enough that you'll just shift your focus to something else and not bother.
I have no doubt the effectiveness will vary greatly depending on type of action, and that it will vary greatly depending on other factors tied to why people wanted to carry out a given action in the first case, of course, but I also think people ask for or do a lot of things where they have not really made a conscious decision that they need it, just decided on a whim that there's no reason not to ask.
Especially with group dynamics involved, it might have been as simple as someone asking a question in a meeting ("do we need the source?" for example) and someone deciding "might as well ask for it, just in case", with nobody actually caring enough to defend it i you stand up to the demand. In that case the need is low enough that even a totally inane excuse for saying no might stand up even if it on the surface looks like a big, important question.
The $50+bn company might even be the potential buyer, but under code escrow they just get it for free (less whatever was negotiated up front.)
I would think the positive of a large client greatly outweighs the liability of the code escrow.
Structure it as 1 year, 2 year, 10 year whatever the hell deal you want. If you are a $50BN company ( you aren't ) you simply fire that customer.
We pointed to (one of many) conditions that said that escrow would be only released to customers who were in good standing with us -before- the escrow event (and that wasn't transferable, as in they couldn't settle accounts with trustees, acquirers or the like - similar to 'not being able to buy retroactive insurance').
After a quick call with a massive customer and walking them through our forecasting strategy and code we saw an abrupt end of communication after that!
Brain rape like something straight out of a silicon valley TV show https://www.youtube.com/watch?v=JlwwVuSUUfc
After giving away our secret sauce they simply cut all communication and one can only assume they are implementing their own version of what we have now...
If they are such a huge customer they should be prepared to pay like everyone else should be if you can prove from your predictions/charts that your algorithms performance is solid.
Give them a short free trial but be careful not to give them too much for free.
We now only offer a 1 day free trial and the value should be obvious after that, start with a crazy price and slowly drip feed discounts, product features and trial extensions like you would market to a normal customer, if they are going to do invest time doing any custom integration with your apis ect then why cant they invest money upfront too?
Its easy as a scientist to not make a strong sales standpoint but your worth more than you think!
I work for a 50,000+ person company. I do software evals all the time, not source code but still. When their sales people call me, I ignore them. I'm very busy, if I want to buy I'll call you.
My group (200+ devs) starts and cancels new projects all the time. No one has time to call every losing vendor and tell them they didn't make the cut, or that we decided not to continue the project.
What do you do if they ask for customizations? What do you do if they request features that are not on your roadmap? What do you do if they ask for a large discount?
How you decide those cases determines how much your business is like a startup moving fast and not negotiating over source code disclosure and how much your business is like an enterprise consultancy that prices out each request for one off work by their clients. There's no right answer. But clarity about how a particular client affects your business model and whether or not that client is worth having right now is important. Are there other potential customers who are easier to service?
It is not clear from the question whether this request is part of ongoing contract negotiations or presented as a prelude to negotiation. If the former, put a price on it. If the latter, it smells a bit like a "no" in the form of a "maybe" or window shopping without much intent or someone higher up with purchasing authority putting on the brakes. Figure out an expected value of the client by assigning probabilities to various size contracts closing at different points in time.
Provide a counter offer. If they want to see your code, counter with "we will show our code when you provide a 10 year contract with X dollars per month/year". Then I would do it.
They are covering their ass, so cover yours. People and businesses who are strong, respect others that are strong.
All my clients will rip me off if I let them. ALL OF THEM. They can't help it. If you asked your cable company "can I have free cable for a year?" And they said "yes", wouldn't you take it? That is what is happening to you. They are testing you.
If you are desperate and foolish, they get your code for free. If you are brave and wise, they get a good partner.
Take a leap of wisdom.
1) The company is not going to acquire you nor will they sue you.
2) They absolutely want to replicate what you've done. It's not so that they can build their own startup and compete with you. It's because they genuinely want to understand how you got the results. Companies are increasingly basing their decisions on ML models and it's not acceptable for them to just "trust your black box". Especially since your model could be based off a misunderstanding of the data.
3) If you decide to say no then chances are they will walk away. Especially in the ML space companies are not going to let their core intellectual property be locked away in someone else's vault. They will simply not work with you.
4) The best way to handle this is simply to get them to sign a NDA or something equivalent that protects you in the case that a rogue employee decides to build a startup him/herself.
I am someone who works in ML for an enterprise company. For god sake give them the code and be grateful they are even doing a PoC with you.
Knowing the sorts of decisions that were made around what types of algorithms and features to use (obvious after a cursory examination of the code) would give them a huge leg-up in developing a similar model.
If you want someone to help specifically advise, guide you through the process, and potentially represent you in negotiations, feel free to get in touch and I can probably help.
By way of background, I primarily work with investors who are selling middle-market software companies to larger companies (over 80 or so personally and oversaw another 80 or so, over the last 3 years). I have a lot of insight and experience into contract negotiation from both sides of the table here. I've also worked for/with the big enterprise software players (SAP, Oracle, Microsoft, etc).
To me it’s clear that there are lots of possible scenarios with lots of possible “right” decisions. Without more info, i doubt anyone can give better advice than what’s on this forum.
Certainly a professional should be able to give better advice but only after they get more info on the specifics.
As a user of data tools myself, I am generally suspicious of any black box model, and would like to understand the model well before using it. For instant if your model is a deep neural network, I'd like to know the structure of the network and the activation of the layers when I run my data through it.
If they're really interested about the guts of the model they will agree to a solution like that. Having the source code will certainly not help them understand this as it is highly unlikely that anyone will dig into this.
Note that asking for the source code is fairly common practice in finance cause people generally distrust black boxes (at a firm I worked for they specifically chose MySQL over some other tools because they had the ability of looking through the code if they needed to).
There is actually no point for them to look at your code...
You can agree to something where if you go bust, then you'll give them the source code.
Escrow arrangements are common to protect clients from software vendors going bust.
1. Ask for a giant pile of money for the privilege.
2. Come up with visualization that answers their question without giving them access to source code. Presumably "how do we know we can trust results" is a common problem.
3. Walk away.
This way you get to jump straight to your exit, and can maybe also structure the deal so that you and your employees get jobs at the BigCo (if you want them). Feel free to set the selling price as high as you like -- they would want the deal more than you would, so you have a lot of leverage in the negotiation.
And if they balk? Hey, they asked for terms to see the source code, you gave them terms! It's not your fault if they don't like them :-D
And everyone here is going on about them wanting to buy the company or acquire licenses to the code. It's just bizarre.
that's great product feedback. and a problem you need to solve. You shouldn't solve it by handing out your source code.
You need to take this approach when dealing with a large potential customer : let's not get into the weeds of what you are asking for, but rather tell me what the underlying need is. That might, for example be : "we'll rely on your code to secure our customers' sensitive data and so we need to take steps to ensure you are practicing industry best processes for security". That need could be addressed by having a third party review your code and processes, protecting your IP.
In your case they said :
"we want to see how your algorithms made their decisions."
For me this would be a "Hello no", perhaps put more politely. You're building a product that has value embodied in those algorithms. The customer is paying you for that service. Therefore you should take absolutely no steps to tell them how it works. There is only downside.
Yes. Satisfy their justification without showing code. It is possible to show how algorithms made their decisions. Make this part of your product. You already noticed there is a demand for it and that delivering a black box can be a deal-breaker. So read up on LIME, decision paths, interpretable models on black box output, etc. and give them the capability to see how an algo made its decision.
> 2) How likely do you think their goal is to genuinely "see what happens under the hood" as opposed to replicate in the future?
Unlikely they'll replicate. It would set them up for legal problems. Depending on how deep your moat is (training data, novel optimization techniques, encoded domain expertise), they probably wouldn't even need to see source to replicate in-house. It may be more about not being hood-winked, paying top dollar for a product that does a few imports from open source libraries.
> 3) Are there any legal protections we can put in place to prevent them from not just copy-pasting our code, but also from "learning from it" or so?
Not that I know of. Perhaps you could charge extra for the code review, so in the case of "learning from it" they'll at least pay for it. A thought exercise: did you learn from open source/open research/commercial solutions before building your Proof-of-Concept? If no, then they don't need to either (provided they can hire the talent), if yes, you are like a thief who is worried they will steal from you :).
Defeat the Confusion: Confidentiality v. Non-Disclosure
Examples of Microsofts Shared Source licensing can be found as well. These contracts are typically reserved for heavy hitters. Who have enhanced security or performance requirements. FBI, JP Morgan, etc. And of course Microsoft has open sourced large portions of its own dev tools and sdks.
Microsoft Shared Source Initiative
I think what you may begin to realize is that its their alternative data that represents the motherload. And its not your algorithms but level of service that will differentiate you. The insights mined from that alternative data may be so valuable as to outweigh your other concerns. And gaining access to it might be the paramount mission for your startup. As the executive, ultimately its your call. Good luck!
Personally, I wouldn't be scared of BigCo ripping you off. For the most part, large companies care a lot about staying on the right side of their contracts, and also it's generally really hard for a large company to out-innovate a startup. So I would be pretty surprised to see them steal your source (assuming you put in place an appropriate NDA etc.). EDIT: I'd be even more surprised to see them try to compete; the worst likely case is that they steal the source and stop paying you, not that they steal the source and get into the data analytics / ML business themselves.
However, I think that in the ML world, this "what the hell is the algorithm doing" question is a really common one, and it'd be super-worthwhile to invest in some sort of tooling to peel back the cover of the algorithm a bit. Validation of appropriate responses against future data is a real quagmire right now, to the point that some people are using ML to help find a solution, but then trying to re-implement the logic more traditionally once the ML algorithms figure out what to design for. I think there's something there, at least for a good subset of use cases.
Also, it's common for a large enterprise to require some sort of source code escrow if they do a big deal with a startup. Sounds like this is different than what they're asking for, since the source in escrow won't be available to them until the escrow conditions are triggered. Again, I wouldn't be concerned about signing escrow agreements, but I would make it a negotiation point, rather than a standard term.
Is it because they're aware that you can't convincingly threaten them with litigation? Do they think you're too small to protect yourself effectively from the danger of IP theft?
If that is the case, then the answer is clear.
If the client actually has legitimate concerns- couldn't they ask you to run some specific tests, or make some experiments, and report the results to them? The amount of time spent to think of such tests should not be more than the amount of time needed to review your code and you could argue that examining the behaviour of your system can be more informative than looking at the source code.
With a startup I was working on, we managed to get meetings with the CEO of the world's largest fashion company and several others with $10b+ revenue. A universal issue, however, was that we were bootstrapped and lacked the funding to insure against their losses, which for them meant unchecked liability.
Regarding meeting them in the middle, I would put together a presentation that describes how your algorithm works at a high level. I'd do your best to split the balance between being transparent and focused on the customer, and not divulging what you consider to be differentiating parts of your implementation.
If you get push back for the above, I think you're dealing with either brain rape, or some pretty unprofessional contacts in your customer's organization. If it's the latter, you should do your best to navigate around those contacts.
Definitely get legal council involved.
A possible way to protect yourself is print it out and put it in old fashioned binders and let them see the binder while you are watching. Not sure if that will fly, but it would be hard(er) for them to steal it. Tell the company your concerns (which are valid) and what methods they would accept. I don't think it's unreasonable for you to bring it up with them.
And in the end, the Macintosh was a huge leap beyond the Alto, that really didn't work much like it at all.
I would say it was a significant iteration, not a leap. iPhone was a leap.
Our experience if I was you is to be weary. Google just wanted to see our secret sauce and once revealed kick us to the curb.
They need to pay you or you walk away!
The OP says they are trying to show a market for their tech by winning this contract. After meeting with Google... all tech company's came out of the wood work and one .. Black & Decker wanted to become a customer. Though they refused to sign any type of NDA and or IP agreement yet were willing to pay a very small fee. My mentor (guy who created Xbox headphones) at the time said if they don't sign anything forget it. Ultimately I did, yet it was the crushing blow to the SpeakerBlast team and me personally for awhile.
I think the guy I met with is still the head of Chrome Audio, yet his boss at the time Regina Dugan was just in the news. She was heading up Facebook's secretive research lab, Building 8.
TL; DR is basically you keep a decision tree in parallel to your model that carries with it long/short-form text that "explains" why the model does what it does.
They may attempt to compete with you or re-implement what you do, but again - if what you did is so good and perhaps non-trivial to re-implement (an assumption), then you should consider the value of what you have above their immediate demands. They might not have to be your "only" customer.
If you are trying to build a business to last on your own, you must ask yourself, is this the kind of customer I want?
Of course, you should consider who the customer is and their mission/vision/values/actions, what your and their goals might be in this instance, how to accommodate their request (like the suggestions for code escrow, etc.) while protecting yourself (surely there are ways through a competent IP lawyer).
Sometimes the big customer can break (or make) your company. I think your course of action depends a lot on what you have and what you want.
That they are interested in you so seriously is probably a good sign, as much as it is something to be concerned about. Consider the value of your use case and what it might be to others as well.
This might be me being naive, but, if they will be good to work with, and a good customer to have, they will be willing to work with you fairly. Otherwise, be cautious and consider the calls for legal advice.
If this understanding is what they're really after, then that's what you need to think about answering. Worrying about ML as an opaque black box is a bit of a thing these days, so it will probably come up with future clients as well.
If you answering this is not satisfying to them, and they keep insisting on the source code, and they can't articulate why, then they are not being honest, and you should walk away (or at least clearly state that if they don't withdraw that requirement, there will be no agreement).
If it is the first, code escrow is very common. You should probably set this up as a gesture of goodwill even if they don't ask for it.
If it is the second, there are tools that you can run to ensure you don't (you should anyway - though the tools tend to be "enterprise software" and thus expensive for what they do). Once you are sure you are free from that type of them a lawyer can draw up legal indemnification documents.
If it is anything else, this is done - for an additional fee. 20 years ago a company sold us an OS, as I recall the price for source code was $100,000 on top of all other costs. My company refused to pay even though it would have saved far more money if we had been able to understand what the code was doing, and thus been able to integrate our code better.
I'm not sure what legal requirements were in place, but you should defiantly have a lawyer who knows this area of law create the agreement. Not just any lawyer, one with experience is worth paying for - find the lawyer first and pay him $200 to give a high end estimate of his costs to draw up the agreement - this is your minimum price for seeing the code. (which is to say you expect to make nothing after the lawyer is paid unless a second customer also wants source code)
Unlike most I wouldn't reject it. However it should be an additional expense, and it should be covered by some strong legal language.
I think the key for you is that it needs to be a third-party company that specializes in these types of code audits. I would NOT just hand over source code to the actual company.
If it is a matter of due diligence, then it should be something that is discussed between your lawyers and the company's lawyers.
If this ask is coming from the engineering side of the company then that is a red flag.
You should also think of the impact in terms of acquiring other customers once you have opened a bit too much to this customer.
The "we want to see how your algorithms made their decisions" justification is a little weird, since that is effectively your secret sauce and just math equations.
Are they worried you're joining in illegally- or unethically-obtained data in making your recommendations? Large enterprise companies have brand reputations that they factor into vendor decisions, and they may not trust you as a startup just yet.
Insofar as their motives are concerned, you should assume the worst; that that they will steal your IP and force you into endless litigation. From a game theoretic min/max perspective this is a sound way to think about it. It's also reasonably likely. There is a good chance they want you to get them started on a problem they don't know how to solve and then they will iterate off your solution. Happens all the time. Imo, if you are building a business in this space, you shouldn't let them do this to you.
Other options: Offer to add some explainable AI visualization stuff into the app. Hide the best parts behind a web service and only agree to give them the source code for the gui. It sounds crazy but people will agree to compromises like this all the time.
They were simply fishing us for answers on how to solve problems.
Unless they are buying your company and are doing due diligence.
You want to avoid a situation where you accommodate them on this, and they come up with some other hoop you have to jump through. Or where they give you some vague excuse like, "thanks, but we decided to go in a different direction" and walk away.
It could also be a bluff to see how far they can push you. Enterprises love to ask for ridiculous things to see if they can get it. I once worked for a fairly large retailer ($10b/yr) that wanted to put into a contract with Microsoft that they could have access to the Windows and SQL Server source code.
We've put source code in escrow to address concerns of going out of business.
But if they really just want to see IP (which is fluid and changes every 90 days), then I'd only provide abstract diagrams and maybe decision tree outputs from the ML.
You know why? It was not secret sauce at all and boolean logic. Amazing how the wool was pulled over a pile of rubbish ;)
1. Everyone involved signs an NDA.
2. No property of the bank is allowed to leave the premises at any point in time. Any supporting documentation was printed out and they were allowed to review it in certain designated rooms. The documents were not allowed to leave that room and they were reviewed to make sure nothing was missing at the end.
3. Any questions must be provided in writing so there is a record of the question and response.
No one gets the secret sauce. They pay for your results.
Because we paid for that model, it belongs to us. Just like how if I pay for an illustration I expect the PSDs.
What? They want "the PSD" before buying.
Honestly, they are no experts, or they wouldn't need you, so them reading and interpreting your code is a patently ridiculous request.
However, there is a huge opportunity here, based on the fact this is new ground for you and them. If they are truly worrying about justifying later the decisions to be made, then you CAN agree to design a report based on whatever your engine is doing, that shows addditional useful data (i.e., not just the correct decision in each situation, but the likelihood of it being correct or expected return). Invent a middle-ground solution.
Then make them agree TO PAY FOR THIS REPORT AS PART OF THE CONTRACT.
Would be good to know if its SaaS or something that goes into their datacenters.
If you are SaaS, I would not share source code. Ever. I often get questions from potential enterprise customers, and while pushing back is not always easy, the reasons are respected. One argument is that you are protecting other customers/tenants by not allowing it, and you will do the same for them when they are onboarded.
If its in their datacenters, there are many reason they might want to see source code (licensing, security, scalability etc). But i would still argue you could keep your core algoritm IP out of that.
Other thoughts: Are they vetting you for potential acquisition?
Because nobody when they are dealing with a vendor is thinking about how to acquire them. They just want your product.
2) Likely they aren't trying to "steal" you IP. They stated their goal as them want to see how your algorithms made their decisions. So just answer as best as you can without giving away your trade secrets.
3) Yes, there is a way to do this with legal protection. Which would be to follow a clean room process. However, this is pretty expensive for a small company to do. So I wouldn't offer it.
If they are bullies, irrational, arrogant or silly, then save yourself lots of headaches and very nicely say its not something you can do.
Have your people interacting with them be confident enough to tactfully state they will walk away from the PoC if that is a condition.
Coca-cola hasn't revealed theirs, why should you?
Ask more questions about if they are seeing things in the analytics results that don't look correct... that might be their concern if they are basing business decisions on your software.
It's not the secret sauce though. The models are the output.
The secret sauce is the engine that builds it and companies aren't asking for the source for that.
I'm also in the DS/ML space, and there is an absolute dearth of explainability in our models. It's atrocious given the decision that are based on these models to not be able to explain why they come up with what they did.
Get back to them and tell them you'll add explainability to your models. Even if it's just something simple like LIME (vary each input to a model, measure the change it produces - works for any model).
2) They are likely looking for concrete methods they can leverage as the platform grows.
3) Don't give them access to the entire kitchen if they're only asking for a recipe.
I can't think of a good reason to require code inspection prior to purchase. Testing, measuring, and generally trying to break the code during evaluation seems fair game though.
I think the onus would be similar for the single customer model. In effect, if you are coding to deliver to them, and they are bankrolling your deployment, they're buying you in all but name. So, the conditionality on their checking of your IPR, should be the same as selling your IPR.
The company asking to see your code has deep pockets. That means they have a whole lot to lose if they breach a contract and steal your intellectual property.
So, if you put together a well-crafted non-disclosure and non-compete contract their risk is high if they mess with you. It'll cost you something to get a solid contract, but it may be worth it if you also get the business.
They could verify it works, that it will preform at scale, and that it doesn't have security vulnerabilities. But I wouldn't want to show that source code to a client who would potentially build the same product themselves.
In these situations it's best to ask yourself a simple question: what would Coca-cola do if they asked to see the formula.
If you hired a photographer would you expect the RAW files ?
Probably if you get then to sign some kind of non-compete non-copy contract and hand over the source then if they do decide to cut you out, you can just sue them and make more money than you would have as a startup anyway.
Get it legalled and then just hope they overstep.
That could include snippets of code where appropriate to show exactly how decisions are being made.
This presentation will likely benefit you in the future.
We are generally not in the business of the services our vendors provide and don’t have the staff, expertise, incentives or instructional fortitude to compete.
That said, I’d never ask for a vendors source code (especially on a POC) unless I felt like an escrow situation was warranted if they went out of business.
If I were you I’d ask for
1. Something like a non-compete or exclusivity agreement. They will not replicate the functionality internally or work with another vendor.
2. A lot of money to see the source code. If they are trying to learn from what you have, then you are providing them with a material benefit you should be compensated for. Offer to throw in consulting services if their objective is to learn.
Bottom line, cover the risk that they steal your magic or otherwise benefit without you being compensated.
Apple borrowed ideas from Xerox. Microsoft were given source code for Mac in order to keep producing Office for it. Both Apple and Microsoft have copied UX elements off of each other.
But I don't recall Microsoft stealing Apple's code. Maybe copying some GUI elements at most...
If they want to look at it badly enough they can buy it.
There are a couple of reasons reasons we might ask to look at your code:
1. While not a reason to look at your code, instead, if we don't have a valid reason to look or don't have access to technical resources either internal or via external consultants who we are fairly confident could build whatever the software is we are buying given time and resources then we DO NOT WANT TO SEE YOUR IP. This goes as far as shell scripts vendors use for stuff that we don't particularly care about. If they leave them on our boxes we make sure we destroy the data. If the company is worth 50bn then there is it a very small chance their about to make a huge pivot to your particular niche and therefore need your code to solve a problem. The reason companies buy software is because they don't want to pay people to maintain it and in addition they DEFINITELY don't want to get sued for looking at your code. So for no other reason than legal repurcussions you can probably trust them not to do anything sketchy. (Disclaimer: small business units do sometimes go rouge. Make sure youre talking to someone who understands the company wide impacts of fucking this up)
2. If the code is going to be used in sensitive environment (ie. Air gapped networks) we may want to scan for both destructive malware dependencies or just bad code that intentionally or unintentionally might damage systems. Also you would be amazed how many vendors build hooks to call out to the internet in standalone software packages that they "certify" for offline use.
3. If we need to build a bunch of integrations ourselves (ie you would be useless to us in so far as needing to understand legacy core banking systems and the like and therefore are not helpful with your knowledge of the code base, we need someone with knowledge of both code bases at a fairly low level) then depending on the size of the code base we might ask for all of it or just all the external interface implementations. Not the definitions. The actual code.
4. If you are a small company it is not unlikely that we will negotiate a clause which says that if you disappear or all your developers die or whatever, then we are allowed to internally use your code base to build our own stuff since we will end up with dependencies on it and will want to make sure we can still function without you (this is obviously not ideal, we would rather throw money at you to make problems go away, but if you aren't a business any more then we just have to hire people to do it) I actually heard a colleague working at a competing bank in Australia tell me that their agreement with hashicorp gives them ownership of consul enterprise code base for use internally if hashicorp disappears. You just need to make sure your lawyers and on this properly to make sure you clearly define the circumstances in which the large companies expectations of you maintaining the code are no longer met and therefore they can do it if they need to.
5. If we just don't trust you to not be hiding some black magic bullshit behind the scenes. This is usually the result of particularly uninformed sales people making claims that cannot technically be true, and thus out due diligence require that we handle it ourselves. It's also much more likely that we will recommend a bunch of software auditing companies we have used and we trust to audit the code base for us, just so we don't have the liability of your IP in our heads.
6. If we have government financial institution regulations which apply to the thing we want to use your software for and we are required to check of sign off the risk. As an example, an Australian bank running things on cloud platforms that hook back into traditional on prem systems it is mandatory without exception that all data at rest or in flight be encrypted. We trusted a large software company on this and only when we had auditors sniffing traffic over the network did we discover that major data intensive operations relating to backup integrity decrypted everything and then pumped it over the wire between instances using HTTP at which point we where $6m deep in licensing fees so we had a few very difficult conversations about "fix it or fuck off and pay us substantial reparations" because we suddenly needed a lot of technical lawyers (who ate as rare as hens teeth) to explain what had happened to avoid fines that could have cost literally billions.
Summary: there are a bunch of reasons a company might want to see your code. If the person you are talking to is speaking on behalf of the whole organisation (ie. They understand broader business implications of doing anything shady) then you're almost definitely safe. If your a bit on the fence about the whole thing, get a third party auditor in, but the request itself is pretty reasonable.
One of the banks owned by our parent company was using a system built by HP to determine loan rates. Now, being a bank that's kind of their core competency so letting someone else build it just seems silly.
But two years later I was on a project trying to hook a bunch of stuff up to this system and we noticed that we couldn't get a proper test run because it kept giving us different figures for the same inputs.
After asking again for the code, or even pseudo code that would explain the behaviour we again told no.
A bunch of lawyers got in a room and they explicitly threatened to sue us if we tried to decompile their code or monitor the system to learn the logic. My response to this was immediate alarm bells because that meant we where not allowed to actually test the system which paid for most of the development of and hosted internally.
A discussion with just our lawyer revealed that, because it was related to home loan rates, our liability was actually significantly reduced if it where technically possible to decompile the code and verify what it was doing because the cost of not being able to explain it to auditors would be so high and the amount we would be sued for might maybe max out in the 10's of millions.
After a bit of very low level analysis we found that their algorithm for determining a load rate included a random number generator to determine an arbitrary discount in the case that very similar data was input repeatedly.
We later discovered that a developer was trying to be clever and get more home lones sold and had entirely missed the point of risk profiling properly. Ie. We don't want to sell you a home loan at a low interest rate if the risk of you defaulting makes it less profitable.
Eventually the resolution we came to was that HP would fix this shit up and they would provide us with all of the original source they had access to.
But for the next 20 years or so we will be carrying the liability of potentiallt hundreds of loans (this system wasn't used by most business units at the time but we where thinking of expanding out its use, hence the project) that have an interest rate that is lower than it needs to be (like 1-1.5% lower!) to cover the risk profile of the person who got the loan.
And all we needed was the code and that wouldn't have happened.
NR developed shake, and when bigger studios started using it, they wanted access to the source code. Disney was one customer who paid to put shake's source code into escrow, with a stipulation that if NR ever went under or got acquired, they could pull it out of escrow and build shake (on Linux) themselves.
Enter Steve Jobs and Apple, who viewed shake as an asset that could help push studios towards Mac OS X, and away from SGI and upstart Linux which was steadily taking SGI's high-end CG market. When Apple acquired Nothing Real, Steve wasn't at all happy with the animation studio's CTO who decided to exercise their contract and pull the shake code out of escrow.
Disney ported shake to amd64 Linux and continued to use and extend it all the way until Tangled (2008), and while it's not their main workhorse compositor these days, shake still runs today, despite Apple's best efforts to kill it ;-). To Apple's credit, they eventually realized that shake's source code was not really that big of an asset, so they offered a deal at one point where studios could pay (50k+?) to get access to the source code. Many studios payed.
Having compared the NR sources to the Apple sources, quite a bit of work was done to put in PPC-specific assembly and performance optimizations (e.g. optimizing for G5's cache sizes) into Darwin-specific #ifdefs.
One takeaway from this story is that the "big" customer (Disney) in this case was not at all interested in stealing IP. They were an animation studio and their core product was something not-software. If the customer in your case is in a closer space, it's a different calculus.
The customer's viewpoint was more of being able to decide their own fate in the event of the company going away. And it turns out, those things did happen, so having the deal in place was a good thing for the customer. Likewise, Nothing Real made out very well in all of this, as they got paid by Disney (among others), and later acquired by Apple who really didn't care enough about the pre-existing contracts to not acquire them.
One protection in the contract was that it only allowed for the studio to produce binaries for themselves, but not distribute them. A practical consequence of this is that Disney could not share shake binaries with Pixar, and while Pixar had also purchased a source license from Apple, Disney could only share (git format-patch) patches with Disney's changes. I really wish we could have open sourced the shake source code, even for historical purposes. Apple still can.
Make sure the deal is reasonable and there shouldn't be any problems. If you don't trust the company then maybe you shouldn't take them on as a client, but otherwise a reputable company will be very willing to keep things as friendly as your lawyers can get them to be. Having a good lawyer is key.
Regarding copy-pasting your code -- in shake's case, having the source code was really helpful for plugin development. Maybe consider ways to help customers leverage actually having the source code. Making the system plugin-able is a great way to do that, as customers mostly want to customize stuff, and plugins are a great way to get customers "deeply integrated" (aka locked) into your ecosystem.