Paul Graham delves into the philosophy of programming, highlighting the benefits of a bottom-up approach as opposed to the traditional top-down design. By building the language to suit the problem at hand, programmers can create code that is clear, concise, and efficient. Graham emphasizes how bottom-up design yields smaller, more agile programs, promotes code re-usability, and makes programs easier to read and modify. This thought-provoking essay challenges the conventional wisdom of software development and encourages programmers to rethink their approach.

1993

(This essay is from the introduction to On Lisp.)
It's a long-standing principle of programming style that the functional
elements of a program should not be too large.  If some component of a
program grows beyond the stage where it's readily comprehensible,
it becomes a mass of complexity which conceals errors as easily
as a big city conceals fugitives.  Such software will be
hard to read, hard to test, and hard to debug.In accordance with this principle, a large program must be divided
into pieces, and the larger the program, the more it must be divided.
How do you divide a program?  The traditional approach is
called top-down design: you say "the purpose of the
program is to do these seven things, so I divide it into seven major
subroutines.  The first subroutine has to do these four things, so
it in turn will have four of its own subroutines," and so on.
This process continues until the whole program has the right level
of granularity-- each part large enough to do something substantial,
but small enough to be understood as a single unit.Experienced Lisp programmers divide up their programs differently.
As well as top-down design, they follow a principle which
could be called bottom-up design-- changing the language
to suit the problem.
In Lisp, you don't just write your program down toward the language,
you also build the language up toward your program.  As you're
writing a program you may think "I wish Lisp had such-and-such an
operator." So you go and write it. Afterward
you realize that using the new operator would simplify the design  
of another part of the program, and so on.
Language and program evolve together.
Like the border between two warring states,
the boundary between language and program is drawn and redrawn,
until eventually it comes to rest along the mountains and rivers,
the natural frontiers of your problem.
In the end your program will look as if the language had been
designed for it.
And when language and
program fit one another well, you end up with code which is
clear, small, and efficient.
It's worth emphasizing that bottom-up design doesn't mean
just writing the same program in a different order.  When you
work bottom-up, you usually end up with a different program.
Instead of a single, monolithic program,
you will get a larger language with more abstract operators,   
and a smaller program written in it.  Instead of a lintel,
you'll get an arch.
In typical code, once you abstract out the parts which are
merely bookkeeping, what's left is much shorter;
the higher you build up the language, the less distance you
will have to travel from the top down to it.
This brings several advantages:
 By making the language do more of the work, bottom-up design
yields programs which are smaller and more agile.  A shorter
program doesn't have to be divided into so many components, and
fewer components means programs which are easier to read or
modify.  Fewer components also means fewer connections between   
components, and thus less chance for errors there.  As
industrial designers strive to reduce the number of moving parts
in a machine, experienced Lisp programmers use bottom-up design
to reduce the size and complexity of their programs. Bottom-up design promotes code re-use.
When you write two
or more programs, many of the utilities you wrote for the first
program will also be useful in the succeeding ones.  Once you've  
acquired a large substrate of utilities, writing a new program can
take only a fraction of the effort it would require if you had to 
start with raw Lisp. Bottom-up design makes programs easier to read.

An instance of this type
of abstraction asks the reader to understand a general-purpose operator;
an instance of functional abstraction asks the reader to understand
a special-purpose subroutine. [1] Because it causes you always to be on the lookout for patterns
in your code, working bottom-up helps to clarify your ideas about
the design of your program.  If two distant components of a program
are similar in form, you'll be led to notice the similarity and
perhaps to redesign the program in a simpler way.


Bottom-up design is possible to a certain degree in languages
other than Lisp.  Whenever you see library functions,
bottom-up design is happening.  However, Lisp gives you much broader
powers in this department, and augmenting the language plays a
proportionately larger role in Lisp style-- so much so that
Lisp is not just a different language, but a whole different way
of programming.It's true that this style of development is better suited to
programs which can be written by small groups.  However, at the
same time, it extends the limits of what can be done by a small
group.  In The Mythical Man-Month,
Frederick Brooks
proposed that the productivity of a group of programmers
does not grow linearly with its size.  As the size of the
group increases, the productivity of individual programmers
goes down.  The experience of Lisp programming  
suggests a more cheerful way
to phrase this law: as the size of the group decreases, the
productivity of individual programmers goes up.
A small group wins, relatively speaking, simply because it's
smaller.  When a small group also takes advantage of the
techniques that Lisp makes possible, it can 
win outright.New: Download On Lisp for Free.
[1] "But no one can read
the program without understanding all your new utilities."
To see why such statements are usually mistaken,
see Section 4.8.

Programming Bottom-Up

Paul Graham delves into the technical advantages of using Lisp in web-based applications. In this insightful essay, Graham highlights key excerpts from a talk he delivered, sharing the specific benefits that he and his team at Viaweb experienced by utilizing Lisp. Whether you're a programmer or simply curious about the inner workings of web development, this article provides a fascinating perspective on the power of Lisp in the world of technology.

After a link to 
Beating the Averages was posted on slashdot, 
some readers wanted to hear in more detail 
about the specific technical advantages we got from using
Lisp in Viaweb.  For those who are interested,
here are some excerpts from a talk I gave in April 2001 at
BBN Labs in Cambridge, MA.

Lisp for Web-Based Applications

Paul Graham delves into the world of startups and challenges the conventional wisdom regarding programming languages. Drawing from his own experiences with Viaweb, Graham explores the advantages of utilizing Lisp over more widely used languages like C++ or Perl. He argues that choosing the right technology can make or break a startup, and encourages entrepreneurs to think outside the box and consider what will truly work best for their unique ventures. A thought-provoking read for anyone interested in the competitive landscape of software development.

Beating the Averages

Paul Graham dives into the intriguing topic of why Java has always seemed suspicious to him and other hackers. While the essay is not a critique of Java itself, it explores the concept of "hacker's radar" and how experienced hackers develop an instinct for identifying good and bad technology. Graham cleverly examines the cover of a book as a metaphor and connects it to his ability to judge technology based on its initial appearance. With insights on ulterior motives, hype, design choices, and user sentiments, this essay offers a thought-provoking analysis of Java's reputation in the programming world.

April 2001This essay developed out of conversations I've had with
several other programmers about why Java smelled suspicious.  It's not
a critique of Java!  It is a case study of hacker's radar.Over time, hackers develop a nose for good (and bad) technology.
I thought it might be interesting to try and write down what
made Java seem suspect to me.Some people who've read this think it's an interesting attempt to write about
something that hasn't been written about before.  Others say I
will get in trouble for appearing to be writing about
things I don't understand.  So, just in
case it does any good, let me clarify that I'm not writing here
about Java (which I have never used) but about hacker's radar
(which I have thought about a lot).The aphorism "you can't tell a book by its cover" originated in
the times when books were sold in plain cardboard covers, to be
bound by each purchaser according to his own taste.  In those days,
you couldn't tell a book by its cover.  But publishing has advanced
since then: present-day publishers work hard to make the cover
something you can tell a book by.I spend a lot of time in bookshops and I feel as if I have by now
learned to understand everything publishers mean to tell me about
a book, and perhaps a bit more.  The time I haven't spent in
bookshops I've spent mostly in front of computers, and I feel as
if I've learned, to some degree, to judge technology by its cover
as well.  It may be just luck, but I've saved myself from a few
technologies that turned out to be real stinkers.So far, Java seems like a stinker to me.  I've never written a Java
program, never more than glanced over reference books about it,
but I have a hunch that it won't be a very successful language.
I may turn out to be mistaken; making predictions about technology
is a dangerous business.  But for what it's worth, as a sort of
time capsule, here's why I don't like the look of Java:
1. It has been so energetically hyped.  Real standards don't have
to be promoted.  No one had to promote C, or Unix, or HTML.  A real
standard tends to be already established by the time most people
hear about it.  On the hacker radar screen, Perl is as big as Java,
or bigger, just on the strength of its own merits.2. It's aimed low.  In the original Java white paper, Gosling
explicitly says Java was designed not to be too difficult for
programmers used to C.  It was designed to be another C++: C plus
a few ideas taken from more advanced languages.  Like the creators
of sitcoms or junk food or package tours, Java's designers were
consciously designing a product for people not as smart as them.
Historically, languages designed for other people to use have been
bad:  Cobol, PL/I, Pascal, Ada, C++.  The good languages have been
those that were designed for their own creators:  C, Perl, Smalltalk,
Lisp.3. It has ulterior motives.  Someone once said that the world would
be a better place if people only wrote books because they had
something to say, rather than because they wanted to write a book.
Likewise, the reason we hear about Java all the time is not because
it has something to say about programming languages.  We hear about
Java as part of a plan by Sun to undermine Microsoft.4. No one loves it.  C, Perl, Python, Smalltalk, and Lisp programmers
love their languages.  I've never heard anyone say that they loved
Java.5. People are forced to use it.  A lot of the people I know using
Java are using it because they feel they have to.  Either it's
something they felt they had to do to get funded, or something they
thought customers would want, or something they were told to do by
management.  These are smart people; if the technology was good,
they'd have used it voluntarily.6. It has too many cooks.  The best programming languages have been
developed by small groups.  Java seems to be run by a committee.
If it turns out to be a good language, it will be the first time
in history that a committee has designed a good language.7. It's bureaucratic.  From what little I know about Java, there
seem to be a lot of protocols for doing things.  Really good
languages aren't like that.  They let you do what you want and get
out of the way.8. It's pseudo-hip.  Sun now pretends that Java is a grassroots,
open-source language effort like Perl or Python.  This one just
happens to be controlled by a giant company.  So the language is
likely to have the same drab clunkiness as anything else that comes
out of a big company.9. It's designed for large organizations.  Large organizations have
different aims from hackers. They want languages that are (believed
to be) suitable for use by large teams of mediocre programmers--
languages with features that, like the speed limiters in U-Haul
trucks, prevent fools from doing too much damage.  Hackers don't
like a language that talks down to them.  Hackers just want power.
Historically, languages designed for large organizations (PL/I,
Ada) have lost, while hacker languages (C, Perl) have won.  The
reason: today's teenage hacker is tomorrow's CTO.10. The wrong people like it.  The programmers I admire most are
not, on the whole, captivated by Java.  Who does like Java?  Suits,
who don't know one language from another, but know that they keep
hearing about Java in the press; programmers at big companies, who
are amazed to find that there is something even better than C++;
and plug-and-chug undergrads, who are ready to like anything that
might get them a job (will this be on the test?).  These people's
opinions change with every wind.11. Its daddy is in a pinch.  Sun's business model is being undermined
on two fronts.  Cheap Intel processors, of the same type used in
desktop machines, are now more than fast enough for servers.  And
FreeBSD seems to be at least as good an OS for servers as Solaris.
Sun's advertising implies that you need Sun servers for industrial
strength applications.  If this were true, Yahoo would be first in
line to buy Suns;  but when I worked there, the servers were all
Intel boxes running FreeBSD.  This bodes ill for Sun's future.  If
Sun runs into trouble, they could drag Java down with them.12. The DoD likes it.  The Defense Department is encouraging
developers to use Java. This seems to me the most damning sign of
all.  The Defense Department does a fine (though expensive) job of
defending the country, but they love plans and procedures and
protocols.  Their culture is the opposite of hacker culture; on
questions of software they will tend to bet wrong.  The last time
the DoD really liked a programming language, it was Ada.
Bear in mind, this is not a critique of Java, but a critique of
its cover.  I don't know Java well enough to like it or dislike
it.  This is just an explanation of why I don't find that I'm eager
to learn it.It may seem cavalier to dismiss a language before you've even tried
writing programs in it.  But this is something all programmers have
to do.  There are too many technologies out there to learn them
all.  You have to learn to judge by outward signs which will be
worth your time.  I have likewise cavalierly dismissed Cobol, Ada,
Visual Basic, the IBM AS400, VRML, ISO 9000, the SET protocol, VMS,
Novell Netware, and CORBA, among others.  They just smelled wrong.It could be that in Java's case I'm mistaken.  It could be that a
language promoted by one big company to undermine another, designed
by a committee for a "mainstream" audience, hyped to the skies,
and beloved of the DoD, happens nonetheless to be a clean, beautiful,
powerful language that I would love programming in.  It could be,
but it seems very unlikely.

Java's Cover

Paul Graham delves into the intriguing question of what makes a programming language popular and explores the influence of hackers in determining a language's success. He argues that a language's popularity is not solely determined by external factors or language designers but rather the opinions and preferences of expert hackers. Graham contends that a good programming language is one that resonates with hackers, catering to their needs and abilities. By examining the dynamics of popularity and the impact of a dedicated user base, Graham presents an insightful perspective on the evolution and significance of programming languages.

Being Popular

In this fascinating article, Paul Graham delves into the origins and essence of Lisp, a programming language that has had a profound impact on the industry. By exploring the pivotal discoveries of John McCarthy, Graham elucidates the distinguishing features of Lisp and its significance in the evolution of programming languages. Discover the unique quality of Lisp that allows it to be written in itself and gain insights into the direction in which languages are heading. Whether you're a seasoned programmer or simply curious about the foundations of computer science, this article offers a captivating glimpse into the world of Lisp.

May 2001

(I wrote this article to help myself understand exactly
what McCarthy discovered.  You don't need to know this stuff
to program in Lisp, but it should be helpful to 
anyone who wants to
understand the essence of Lisp  both in the sense of its
origins and its semantic core.  The fact that it has such a core
is one of Lisp's distinguishing features, and the reason why,
unlike other languages, Lisp has dialects.)In 1960, John 
McCarthy published a remarkable paper in
which he did for programming something like what Euclid did for
geometry. He showed how, given a handful of simple
operators and a notation for functions, you can
build a whole programming language.
He called this language Lisp, for "List Processing,"
because one of his key ideas was to use a simple
data structure called a list for both
code and data.It's worth understanding what McCarthy discovered, not
just as a landmark in the history of computers, but as
a model for what programming is tending to become in
our own time.  It seems to me that there have been
two really clean, consistent models of programming so
far: the C model and the Lisp model.
These two seem points of high ground, with swampy lowlands
between them.  As computers have grown more powerful,
the new languages being developed have been moving
steadily toward the Lisp model.  A popular recipe
for new programming languages in the past 20 years 
has been to take the C model of computing and add to
it, piecemeal, parts taken from the Lisp model,
like runtime typing and garbage collection.In this article I'm going to try to explain in the
simplest possible terms what McCarthy discovered.
The point is not just to learn about an interesting
theoretical result someone figured out forty years ago,
but to show where languages are heading.
The unusual thing about Lisp  in fact, the defining
quality of Lisp  is that it can be written in
itself.  To understand what McCarthy meant by this,
we're going to retrace his steps, with his mathematical
notation translated into running Common Lisp code.

The Roots of Lisp

Paul Graham delves into the fascinating world of programming language design, highlighting key insights from his panel discussion at MIT. He emphasizes that programming languages are ultimately for people, aiming to alleviate the complexities and burdens of machine language. Graham explores the balance between mathematical elegance and catering to human weaknesses, urging designers to consider both pragmatic and aesthetic factors. He also emphasizes the importance of designing languages to suit the needs of the best programmers, giving them control and fostering brevity. Discover the intriguing dynamics behind language design through Graham's thought-provoking essay.

May 2001

(These are some notes I made
for a panel discussion on programming language design
at MIT on May 10, 2001.)1. Programming Languages Are for People.Programming languages
are how people talk to computers.  The computer would be just as
happy speaking any language that was unambiguous.  The reason we
have high level languages is because people can't deal with
machine language.  The point of programming
languages is to prevent our poor frail human brains from being 
overwhelmed by a mass of detail.Architects know that some kinds of design problems are more personal
than others.  One of the cleanest, most abstract design problems
is designing bridges.  There your job is largely a matter of spanning
a given distance with the least material.  The other end of the
spectrum is designing chairs.  Chair designers have to spend their
time thinking about human butts.Software varies in the same way. Designing algorithms for routing
data through a network is a nice, abstract problem, like designing
bridges.  Whereas designing programming languages is like designing
chairs: it's all about dealing with human weaknesses.Most of us hate to acknowledge this.  Designing systems of great
mathematical elegance sounds a lot more appealing to most of us
than pandering to human weaknesses.  And there is a role for mathematical
elegance: some kinds of elegance make programs easier to understand.
But elegance is not an end in itself.And when I say languages have to be designed to suit human weaknesses,
I don't mean that languages have to be designed for bad programmers.
In fact I think you ought to design for the 
best programmers, but
even the best programmers have limitations.  I don't think anyone
would like programming in a language where all the variables were
the letter x with integer subscripts.2. Design for Yourself and Your Friends.If you look at the history of programming languages, a lot of the best
ones were languages designed for their own authors to use, and a
lot of the worst ones were designed for other people to use.When languages are designed for other people, it's always a specific
group of other people: people not as smart as the language designer.
So you get a language that talks down to you.  Cobol is the most
extreme case, but a lot of languages are pervaded by this spirit.It has nothing to do with how abstract the language is.  C is pretty
low-level, but it was designed for its authors to use, and that's
why hackers like it.The argument for designing languages for bad programmers is that
there are more bad programmers than good programmers.  That may be
so.  But those few good programmers write a disproportionately
large percentage of the software.I'm interested in the question, how do you design a language that
the very best hackers will like?  I happen to think this is
identical to the question, how do you design a good programming
language?, but even if it isn't, it is at least an interesting
question.3. Give the Programmer as Much Control as Possible.Many languages
(especially the ones designed for other people) have the attitude
of a governess: they try to prevent you from
doing things that they think aren't good for you.  I like the   
opposite approach: give the programmer as much
control as you can.When I first learned Lisp, what I liked most about it was
that it considered me an equal partner.  In the other languages
I had learned up till then, there was the language and there was my   
program, written in the language, and the two were very separate.
But in Lisp the functions and macros I wrote were just like those
that made up the language itself.  I could rewrite the language
if I wanted.  It had the same appeal as open-source software.4. Aim for Brevity.Brevity is underestimated and even scorned.
But if you look into the hearts of hackers, you'll see that they
really love it.  How many times have you heard hackers speak fondly
of how in, say, APL, they could do amazing things with just a couple
lines of code?  I think anything that really smart people really
love is worth paying attention to.I think almost anything
you can do to make programs shorter is good.  There should be lots
of library functions; anything that can be implicit should be;
the syntax should be terse to a fault; even the names of things
should be short.And it's not only programs that should be short.  The manual should
be thin as well.  A good part of manuals is taken up with clarifications
and reservations and warnings and special cases.  If you force  
yourself to shorten the manual, in the best case you do it by fixing
the things in the language that required so much explanation.5. Admit What Hacking Is.A lot of people wish that hacking was
mathematics, or at least something like a natural science.  I think
hacking is more like architecture.  Architecture is
related to physics, in the sense that architects have to design
buildings that don't fall down, but the actual goal of architects
is to make great buildings, not to make discoveries about statics.What hackers like to do is make great programs.
And I think, at least in our own minds, we have to remember that it's
an admirable thing to write great programs, even when this work 
doesn't translate easily into the conventional intellectual
currency of research papers.  Intellectually, it is just as
worthwhile to design a language programmers will love as it is to design a
horrible one that embodies some idea you can publish a paper
about.1. How to Organize Big Libraries?Libraries are becoming an
increasingly important component of programming languages.  They're
also getting bigger, and this can be dangerous.  If it takes longer
to find the library function that will do what you want than it
would take to write it yourself, then all that code is doing nothing
but make your manual thick.  (The Symbolics manuals were a case in 
point.)  So I think we will have to work on ways to organize
libraries.  The ideal would be to design them so that the programmer
could guess what library call would do the right thing.2. Are People Really Scared of Prefix Syntax?This is an open
problem in the sense that I have wondered about it for years and
still don't know the answer.  Prefix syntax seems perfectly natural
to me, except possibly for math.  But it could be that a lot of 
Lisp's unpopularity is simply due to having an unfamiliar syntax.   
Whether to do anything about it, if it is true, is another question. 

3. What Do You Need for Server-Based Software?

I think a lot of the most exciting new applications that get written
in the next twenty years will be Web-based applications, meaning
programs that sit on the server and talk to you through a Web
browser.  And to write these kinds of programs we may need some
new things.One thing we'll need is support for the new way that server-based 
apps get released.  Instead of having one or two big releases a
year, like desktop software, server-based apps get released as a
series of small changes.  You may have as many as five or ten
releases a day.  And as a rule everyone will always use the latest
version.You know how you can design programs to be debuggable?
Well, server-based software likewise has to be designed to be
changeable.  You have to be able to change it easily, or at least
to know what is a small change and what is a momentous one.Another thing that might turn out to be useful for server based
software, surprisingly, is continuations.  In Web-based software
you can use something like continuation-passing style to get the
effect of subroutines in the inherently 
stateless world of a Web
session.  Maybe it would be worthwhile having actual continuations,
if it was not too expensive.4. What New Abstractions Are Left to Discover?I'm not sure how
reasonable a hope this is, but one thing I would really love to    
do, personally, is discover a new abstraction-- something that would
make as much of a difference as having first class functions or
recursion or even keyword parameters.  This may be an impossible
dream.  These things don't get discovered that often.  But I am always
looking.1. You Can Use Whatever Language You Want.Writing application
programs used to mean writing desktop software.  And in desktop
software there is a big bias toward writing the application in the
same language as the operating system.  And so ten years ago,
writing software pretty much meant writing software in C.
Eventually a tradition evolved:
application programs must not be written in unusual languages.  
And this tradition had so long to develop that nontechnical people
like managers and venture capitalists also learned it.Server-based software blows away this whole model.  With server-based
software you can use any language you want.  Almost nobody understands
this yet (especially not managers and venture capitalists).
A few hackers understand it, and that's why we even hear
about new, indy languages like Perl and Python.  We're not hearing
about Perl and Python because people are using them to write Windows
apps.What this means for us, as people interested in designing programming
languages, is that there is now potentially an actual audience for
our work.2. Speed Comes from Profilers.Language designers, or at least
language implementors, like to write compilers that generate fast
code.  But I don't think this is what makes languages fast for users.
Knuth pointed out long ago that speed only matters in a few critical
bottlenecks.  And anyone who's tried it knows that you can't guess
where these bottlenecks are.  Profilers are the answer.Language designers are solving the wrong problem.  Users don't need
benchmarks to run fast.  What they need is a language that can show
them what parts of their own programs need to be rewritten.  That's
where speed comes from in practice.  So maybe it would be a net 
win if language implementors took half the time they would
have spent doing compiler optimizations and spent it writing a
good profiler instead.3. You Need an Application to Drive the Design of a Language.This may not be an absolute rule, but it seems like the best languages
all evolved together with some application they were being used to
write.  C was written by people who needed it for systems programming.
Lisp was developed partly to do symbolic differentiation, and
McCarthy was so eager to get started that he was writing differentiation
programs even in the first paper on Lisp, in 1960.It's especially good if your application solves some new problem.
That will tend to drive your language to have new features that   
programmers need.  I personally am interested in writing
a language that will be good for writing server-based applications.[During the panel, Guy Steele also made this point, with the
additional suggestion that the application should not consist of
writing the compiler for your language, unless your language
happens to be intended for writing compilers.]4. A Language Has to Be Good for Writing Throwaway Programs.You know what a throwaway program is: something you write quickly for
some limited task.  I think if you looked around you'd find that  
a lot of big, serious programs started as throwaway programs.  I
would not be surprised if most programs started as throwaway
programs.  And so if you want to make a language that's good for
writing software in general, it has to be good for writing throwaway
programs, because that is the larval stage of most software.5. Syntax Is Connected to Semantics.It's traditional to think of
syntax and semantics as being completely separate.  This will
sound shocking, but it may be that they aren't.
I think that what you want in your language may be related
to how you express it.I was talking recently to Robert Morris, and he pointed out that
operator overloading is a bigger win in languages with infix
syntax.  In a language with prefix syntax, any function you define
is effectively an operator.  If you want to define a plus for a
new type of number you've made up, you can just define a new function
to add them.  If you do that in a language with infix syntax,
there's a big difference in appearance between the use of an
overloaded operator and a function call.1. New Programming Languages.Back in the 1970s
it was fashionable to design new programming languages.  Recently
it hasn't been.  But I think server-based software will make new  
languages fashionable again.  With server-based software, you can
use any language you want, so if someone does design a language that
actually seems better than others that are available, there will be
people who take a risk and use it.2. Time-Sharing.Richard Kelsey gave this as an idea whose time
has come again in the last panel, and I completely agree with him.
My guess (and Microsoft's guess, it seems) is that much computing
will move from the desktop onto remote servers.  In other words,  
time-sharing is back.  And I think there will need to be support
for it at the language level.  For example, I know that Richard
and Jonathan Rees have done a lot of work implementing process  
scheduling within Scheme 48.3. Efficiency.Recently it was starting to seem that computers
were finally fast enough.  More and more we were starting to hear
about byte code, which implies to me at least that we feel we have
cycles to spare.  But I don't think we will, with server-based
software.   Someone is going to have to pay for the servers that
the software runs on, and the number of users they can support per
machine will be the divisor of their capital cost.So I think efficiency will matter, at least in computational
bottlenecks.  It will be especially important to do i/o fast,
because server-based applications do a lot of i/o.It may turn out that byte code is not a win, in the end.  Sun and
Microsoft seem to be facing off in a kind of a battle of the byte
codes at the moment.  But they're doing it because byte code is a
convenient place to insert themselves into the process, not because
byte code is in itself a good idea.  It may turn out that this
whole battleground gets bypassed.  That would be kind of amusing.1. Clients.This is just a guess, but my guess is that
the winning model for most applications will be purely server-based.
Designing software that works on the assumption that everyone will 
have your client is like designing a society on the assumption that
everyone will just be honest.  It would certainly be convenient, but
you have to assume it will never happen.I think there will be a proliferation of devices that have some
kind of Web access, and all you'll be able to assume about them is
that they can support simple html and forms.  Will you have a
browser on your cell phone?  Will there be a phone in your palm  
pilot?  Will your blackberry get a bigger screen? Will you be able
to browse the Web on your gameboy?  Your watch?  I don't know.  
And I don't have to know if I bet on
everything just being on the server.  It's
just so much more robust to have all the 
brains on the server.2. Object-Oriented Programming.I realize this is a
controversial one, but I don't think object-oriented programming
is such a big deal.  I think it is a fine model for certain kinds
of applications that need that specific kind of data structure,   
like window systems, simulations, and cad programs.  But I don't
see why it ought to be the model for all programming.I think part of the reason people in big companies like object-oriented
programming is because it yields a lot of what looks like work.
Something that might naturally be represented as, say, a list of
integers, can now be represented as a class with all kinds of
scaffolding and hustle and bustle.Another attraction of
object-oriented programming is that methods give you some of the
effect of first class functions.  But this is old news to Lisp
programmers.  When you have actual first class functions, you can
just use them in whatever way is appropriate to the task at hand,
instead of forcing everything into a mold of classes and methods.What this means for language design, I think, is that you shouldn't
build object-oriented programming in too deeply.  Maybe the
answer is to offer more general, underlying stuff, and let people design
whatever object systems they want as libraries.3. Design by Committee.Having your language designed by a committee is a big pitfall,  
and not just for the reasons everyone knows about.  Everyone
knows that committees tend to yield lumpy, inconsistent designs.  
But I think a greater danger is that they won't take risks.
When one person is in charge he can take risks
that a committee would never agree on.Is it necessary to take risks to design a good language though?
Many people might suspect
that language design is something where you should stick fairly
close to the conventional wisdom.  I bet this isn't true.
In everything else people do, reward is proportionate to risk.
Why should language design be any different?

Five Questions about Language Design

Paul Graham delves into the future of software and the rise of server-based applications in this insightful and forward-thinking essay. He argues that this shift towards web-based software will revolutionize how programmers work and open doors for startups to thrive in this new landscape. Graham explores the advantages of this model, including ease of use, cost-effectiveness, mobility, and reliability. Join him as he takes you on a journey through the early days of this technological shift and provides fascinating insights into the potential of this emerging trend.

The Other Road Ahead

In this thought-provoking essay, Paul Graham delves into the revolutionary ideas that made Lisp stand out from other programming languages. From conditionals and recursion to garbage collection and the composition of expressions, Graham explores the unique features that have shaped the language's evolution. By examining Lisp's distinct concepts, he sheds light on how it has influenced modern programming practices and even inspired elements in other languages. Delve into this captivating journey to discover the origins of Lisp and its ongoing impact on the programming landscape.

December 2001 (rev. May 2002)

(This article came about in response to some questions on
the LL1 mailing list.  It is now
incorporated in Revenge of the Nerds.)When McCarthy designed Lisp in the late 1950s, it was
a radical departure from existing languages,
the most important of which was Fortran.Lisp embodied nine new ideas:
1. Conditionals.  A conditional is an if-then-else
construct.  We take these for granted now.  They were 
invented
by McCarthy in the course of developing Lisp. 
(Fortran at that time only had a conditional
goto, closely based on the branch instruction in the 
underlying hardware.)  McCarthy, who was on the Algol committee, got
conditionals into Algol, whence they spread to most other
languages.2. A function type. In Lisp, functions are first class 
objects-- they're a data type just like integers, strings,
etc, and have a literal representation, can be stored in variables,
can be passed as arguments, and so on.3. Recursion.  Recursion existed as a mathematical concept
before Lisp of course, but Lisp was the first programming language to support
it.  (It's arguably implicit in making functions first class
objects.)4. A new concept of variables.  In Lisp, all variables
are effectively pointers. Values are what
have types, not variables, and assigning or binding
variables means copying pointers, not what they point to.5. Garbage-collection.6. Programs composed of expressions. Lisp programs are 
trees of expressions, each of which returns a value.  
(In some Lisps expressions
can return multiple values.)  This is in contrast to Fortran
and most succeeding languages, which distinguish between
expressions and statements.It was natural to have this
distinction in Fortran because (not surprisingly in a language
where the input format was punched cards) the language was
line-oriented.  You could not nest statements.  And
so while you needed expressions for math to work, there was
no point in making anything else return a value, because
there could not be anything waiting for it.This limitation
went away with the arrival of block-structured languages,
but by then it was too late. The distinction between
expressions and statements was entrenched.  It spread from 
Fortran into Algol and thence to both their descendants.When a language is made entirely of expressions, you can
compose expressions however you want.  You can say either
(using Arc syntax)(if foo (= x 1) (= x 2))or(= x (if foo 1 2))7. A symbol type.  Symbols differ from strings in that
you can test equality by comparing a pointer.8. A notation for code using trees of symbols.9. The whole language always available.  
There is
no real distinction between read-time, compile-time, and runtime.
You can compile or run code while reading, read or run code
while compiling, and read or compile code at runtime.Running code at read-time lets users reprogram Lisp's syntax;
running code at compile-time is the basis of macros; compiling
at runtime is the basis of Lisp's use as an extension
language in programs like Emacs; and reading at runtime
enables programs to communicate using s-expressions, an
idea recently reinvented as XML.
When Lisp was first invented, all these ideas were far
removed from ordinary programming practice, which was
dictated largely by the hardware available in the late 1950s.Over time, the default language, embodied
in a succession of popular languages, has
gradually evolved toward Lisp.  1-5 are now widespread.
6 is starting to appear in the mainstream.
Python has a form of 7, though there doesn't seem to be
any syntax for it.  
8, which (with 9) is what makes Lisp macros
possible, is so far still unique to Lisp,
perhaps because (a) it requires those parens, or something 
just as bad, and (b) if you add that final increment of power, 
you can no 
longer claim to have invented a new language, but only
to have designed a new dialect of Lisp ; -)Though useful to present-day programmers, it's
strange to describe Lisp in terms of its
variation from the random expedients other languages
adopted.  That was not, probably, how McCarthy
thought of it.  Lisp wasn't designed to fix the mistakes
in Fortran; it came about more as the byproduct of an
attempt to axiomatize computation.

What Made Lisp Different

In this provocative essay, Paul Graham challenges the prevailing obsession with object-oriented programming. Drawing from his own experiences and insights, he questions the necessity of object-oriented programming in every aspect of coding. Graham dives into the reasons why people are drawn to it, debunking some common misconceptions along the way. With a hint of humor, he questions the bloated code and duplication that often result from this approach. If you're curious about alternative programming styles or simply want to challenge the status quo, this article is a must-read.

There is a kind of mania for object-oriented programming at the moment, but

some of the smartest programmers I know are some of the least excited about it.My own feeling is that object-oriented
programming is a useful technique in some
cases, but it isn't something that has to pervade every program you
write.  You should be able to define new types,
but you shouldn't have to express every program as the
definition of new types.I think there are five reasons people like object-oriented   
programming, and three and a half of them are bad:
 Object-oriented programming is exciting   
if you have a statically-typed language without 
lexical closures or macros.  To some degree, it offers a way around these
limitations.  (See Greenspun's Tenth Rule.) Object-oriented programming is popular in big companies,
because it suits the way they write software.  At big companies,
software tends to be written by large (and frequently changing)  
teams of
mediocre programmers.  Object-oriented programming imposes a
discipline on these programmers that prevents any one of them
from doing too much damage.  The price is that the resulting
code is bloated with protocols and full of duplication.  
This is not too high a price for big companies, because their
software is probably going to be bloated and full of 
duplication anyway. Object-oriented
programming generates a lot of what looks like work.
Back in the days of fanfold, there was a type of programmer who
would only put five or ten lines of code on a page, preceded
by twenty lines of elaborately formatted comments. 
Object-oriented programming is like crack for these people: it lets
you incorporate all this scaffolding right into your source
code.  Something that a Lisp hacker might handle by pushing
a symbol onto a list becomes a whole file of classes and
methods.  So it is a good tool if you want to convince yourself,
or someone else, that you are doing a lot of work. If a language is itself an object-oriented program, it can
be extended by users.  Well, maybe.  Or maybe you can do
even better by offering the sub-concepts
of object-oriented programming a la carte.  Overloading, 
for example, is not intrinsically tied to classes.  We'll see. Object-oriented abstractions map neatly onto the domains
of  certain specific kinds of programs, like simulations and CAD
systems.                                        


I personally have never needed object-oriented abstractions.
Common Lisp has an enormously powerful object system and I've
never used it once.  I've done a lot of things (e.g. making         
hash tables full of closures) that would have required 
object-oriented techniques to do in wimpier languages, but
I have never had to use CLOS.Maybe I'm just stupid, or have worked on some limited subset
of applications.  There is a danger in designing a language
based on one's own experience of programming.  But it seems
more dangerous to put stuff in that you've never needed 
because it's thought to be a good idea.

Read Paul Graham

Choose Your Reading Path

Full Backlog

Best Hits

You'll start with:

Programming Bottom-Up

Lisp for Web-Based Applications

Beating the Averages

About Paul Graham