Web programming for programmers who aren't web programmers

Dragonmaster Lou

So I'm thinking about learning some web programming stuff just for the heck of it -- partly to just branch out my overall programming expertise and partly to help my wife out when she works on web designs (she's a graphic/web designer, not a programmer). However, I am a systems programmer, not a web programmer. Most of what I work on is lower-level stuff in C and C++ (with a little bit of Java on occasion) and, in general, once you install the software I write you will only notice it's there when it's not working and it tends to be background server stuff.

Anyway, learning the languages typically used for web programming isn't an issue -- I'm already reasonably comfortable with Python and can pick up JavaScript (probably with a good browser abstraction library like Jquery or something), PHP, or whatever other languages I need to know without too much difficulty. I'm also pretty handy at setting up a basic Linux server and can also pick up anything else I need to know for a non-stock web server installation without that much difficulty either.

What I'm mostly interested in is a guide to what to do and what not to do for the beginning web programmer -- best practices, things to avoid, tips for avoiding potential security issues, and so on. Ideally, I'd want something aimed at someone who already knows programming. Any recommendations or pearls of wisdom from your own experiences?

Apreche

Basically all you do is write a program that takes an HTTP request as input and returns an HTTP response as output.

The only two major security things to watch out for are injection attacks and cross site scripting/request forgeries.

For performance the key is mostly to reduce the number of SQL queries per page load and make those queries as fast as possible. The database is almost always the bottleneck unless you do something weird.

Dragonmaster Lou

Okay, the HTTP request/response stuff, yeah, I've dabbled a bit with that in the past, so that's no real problem.

I guess I'll be reading up on injection and xss attacks/forgeries to find out how they tend to slip into your system. I'll also need to brush up on my SQL as I only have a very basic knowledge of it. Thanks.

dsf

from a low level perspective I think you should take a look at socket programming in your language of choice. From the high level you have PHP and it's frameworks, ASP.NET, JavaScript and it's frameworks, CSS to make it easier to make it pretty, and tons of other stuff. Start with PHP, JavaScript and CSS and go from there. Create something using PHP(php is server-side), the use CSS and JavaScript to add bells and whistles(client-side). Most of the other stuff are frameworks that are used on a higher level. Of course Scott is a the senior programmer here I would differ to his info. But sometimes I find that people who are programming on a really high level forget how to break things down to non-programmers/starting-programmers.

edit: think of SQL as something to be encapsulated in another language.

Apreche

edit: think of SQL as something to be encapsulated in another language.

This is the number one source of fail these days. Everyone uses ORMs to rather than learning SQL. ORMs are great, I use them all the time. However, if you use one without understanding relational databases and SQL, you are in for a rude awakening. They can make life easier for people who do know those things, but they will bring a world of hidden terrors for those who do not.

Dragonmaster Lou

from a low level perspective I think you should take a look at socket programming in your language of choice. From the high level you have PHP and it's frameworks, ASP.NET, JavaScript and it's frameworks, CSS to make it easier to make it pretty, and tons of other stuff. Start with PHP, JavaScript and CSS and go from there. Create something using PHP(php is server-side), the use CSS and JavaScript to add bells and whistles(client-side). Most of the other stuff are frameworks that are used on a higher level. Of course Scott is a the senior programmer here I would differ to his info. But sometimes I find that people who are programming on a really high level forget how to break things down to non-programmers/starting-programmers.

Socket programming is no big deal -- I've been doing it professionally off and on in C since I got out of school. I was thinking more at the higher level -- leaving the socket stuff to Apache (I don't want to reinvent the wheel and write my own custom HTTP server unless I absolutely have to) and concentrating more on the higher level stuff.

edit: think of SQL as something to be encapsulated in another language.

This is the number one source of fail these days. Everyone uses ORMs to rather than learning SQL. ORMs are great, I use them all the time. However, if you use one without understanding relational databases and SQL, you are in for a rude awakening. They can make life easier for people who do know those things, but they will bring a world of hidden terrors for those who do not.

I agree. I've noticed way too much wackiness in various ORMs I've come across at my various jobs over the years. It also looks kinda clunky and it is kind of lame when you need a couple of dozen lines of ORM calls to do something with crappy performance that can be done in a single well-written custom SQL statement.

I'm more of the opinion that while ORMs aren't bad, per se, you need to know enough about SQL such that when your ORM is flaking out on you, you can read the SQL its producing to make sure it's actually what you want and/or replace it with your own hand-written custom SQL as necessary.

Apreche

I think a good place for you to start might be Flask.

http://flask.pocoo.org/
It's a Python micro-framework that lets you attach functions to URL patterns. Then when the web server gets hit at a matching URL, that function is executed and its return value is sent as the Http Response. It's a good way to learn web programming without going too low or too high a level. Once you get the idea, you can move onto a full framework like Django or whatever without hurting yourself. I tend to use Jinja templates with Flask. http://jinja.pocoo.org/docs/

dsf

oh I know SQL, I'm not at the level I want to be at with, but I do know it. I'm just saying that you need to wrap up SQL statements in strings and concatenate them together in within an external program in order to get the Statement you need to perform a DB task. Let's not nit pick too much. I'm at the point where I am just starting to work with stored procedures and triggers.

Dragonmaster Lou

Ah, I'll take a look at Flask, but based on your description it sounds like the kind of thing I'm looking for to get started with. Thanks!

dsf

but all in all, Listen to Scott. When I have question I go to him.

But that block quote you made a comment about from my earlier post had some relevant info other than socket programming.

Apreche

I'm at the point where I am just starting to work with stored procedures and triggers.

I'm actually not a fan of stored procedures and triggers for anything but big enterprise applications. They do work, and they are good for performance. However, what happens when you have a stored procedure is that a key piece of logic is missing from your codebase. If someone takes the code for your application and runs it on a database without those stored procedures, they are going to have problems. Likewise, if you forget your stored procedures, you are also in trouble. I dislike cron jobs for the same reason. Cron is great for scheduled system tasks, like log rotation, but I don't like it for application tasks.

Instead what I do is use an AMQP message queue. Specifically I use rabbitmq and celery and django-celery. This allows me to put scheduled tasks as well as tasks with delayed execution right into my codebase directly. For example, when I upload an mp3 to frontrowcrew.com there is a celery job that sends it to Libsyn via FTP. Likewise, the automatic tweets of daily episodes are a scheduled celery job.

dsf

Instead what I do is use an AMQP message queue. Specifically I use rabbitmq and celery and django-celery. This allows me to put scheduled tasks as well as tasks with delayed execution right into my codebase directly. For example, when I upload an mp3 to frontrowcrew.com there is a celery job that sends it to Libsyn via FTP. Likewise, the automatic tweets of daily episodes are a scheduled celery job.

I'll take a look at that.

Anthony Heman

A lot of this depends on what exactly you're going to be implementing. If your wife is already competent with html/css, and you're mostly just writing javascript for display and scripts to run to retrieve information you really only need to focus on javascript and php/python. If she just builds webpage mock-ups to do slices and actually implement, then it gets into all kinds of extra complexity.

Apreche

Also, unless you are doing something that is genuinely new, it is often best to avoid web programming altogether. If you can do it with Wordpress, don't write it yourself. There's no way no matter how much work you do that you can solve all the problems that WordPress has already solved for you. Using it, or any other out of the box solution, will turn all your programming problems into IT problems.

Dragonmaster Lou

But that block quote you made a comment about from my earlier post had some relevant info other than socket programming.

Yeah, but a lot of it consisted of languages and frameworks to look into and I'm not quite as interested in that, at least not yet. The basic ideas behind it aren't all that different from what I'd do to approach learning any new language/technology/etc. -- think up a "warm up" project, select the tools/languages I wish to implement it in, study those tools, and let 'er rip. Remember, I'm not a newbie programmer -- I'm a newbie web programmer who has already done some pretty significant coding -- just not anything with a web-based (or any, really -- remember, I program servers and drivers here) front-end and a database back-end. I'm more interested in more general concepts, such as "how do I avoid injection attacks," "how do I write secure web code," "where to check for performance problems," "how to properly get your AJAX front end talking to your back-end," and so on. After looking at those concepts, then I can figure out/research how to implement the solutions in my languages/frameworks of choice. Of course, suggestions of good starter frameworks, like Flask, are always appreciated.

A lot of this depends on what exactly you're going to be implementing. If your wife is already competent with html/css, and you're mostly just writing javascript for display and scripts to run to retrieve information you really only need to focus on javascript and php/python. If she just builds webpage mock-ups to do slices and actually implement, then it gets into all kinds of extra complexity.

She's very good with HTML and CSS (she actually writes it all by hand and doesn't use any GUI web design tools except for their live preview functionality), but it's the JavaScript and PHP/Python stuff she's lacking on, so that's the direction I want to come in from to help her out.

Anthony Heman

Regarding security, there are a lot of details, more than I know. And what you need to implement depends on how exactly you write your code and what your dealing with. Step one is generally data validation. Don't trust anything from the user unless it matches your white-list. Then treat all data for potential exceptions even above and beyond that. I'm going to guess you're already super-familiar with regular expressions, so you should apply that where possible.

Another concern is what the website is used for. Generally I tell everyone to build with HTTPS in mind from the ground up now. If you're doing anything with credit cards it's required, but even if your just submitting usernames and passwords I would recommend getting a valid certificate and using it for everything that isn't a brochure site.

If you're doing anything with sessions or cookies (and sessions store a cookie so that's important too), you should read into what practices are used to protect yourself from somebody spoofing/duping/predicting your session IDs or cookie information. Even big name websites I've found have flaws in this system. I know that while I was signing up for my PAX hotel room last year, I actually was put into some other users session and could see their registration information because (from what I could figure) they assigned me a session ID that belonged to someone else while it was still active. This is one of those areas where you'll never be 100% secure, but you can make it reasonably unlikely that anyone will abuse it.

Regarding the back-end, you may want to consider isolating the ajax layer from the stuff you want to protect. On my current project, we're building something heavily ajax-reliant. In order to make it more secure, the ajax interface only ever talks to an api we've been building that exists on a sort of middle server. The middle server is firewalled off from everything but the webserver, and has only the functionality we allow it to interact with our back-end. Not something you'll need to do immediately, but this gives us some extra protection so that if someone with poor practices (or harmful intent) leaves a giant open security hole on the webserver, the damage is mitigated since it can only abuse functionality that we've decided would be acceptable for the website to interact with. The webserver has its own sql database for some actions, but even if someone gained total control over it they wouldn't have access (hopefully) to anything but what the api server allows.

Meh, and these are just my pet issues. Doesn't even scratch the surface really. XSS attacks are a concern, how to store/setup user passwords is a sort of issue, your specific security concerns concerning a specific websites requirements (PCI compliance, telecommunications laws, banking).

dsf

Well, I understand what you're saying, but I don't think you'll be able to do what you want to do without understanding PHP, Javascript and SQL. That's pretty much as fundamental as you can get. There is really no short cut around it. Security and the frameworks are kind of assuming you have at least a base level of proficiency with those things. For example, Data validation. You can make this really beautiful JavaScript validation tool only to have it defeated by a user that has JavaScript disabled. So you need to do the processing serverside or your security is defeated. But once you have the validation on the serverside working you can then have JavaScript do most of the grunt work on the client side and then have the server side validation there to catch it just in case. You are pretty much talking out of your ass about SQL injection attacks if you don't know SQL or the programming(PHP, ASP.NET ect.) to put together a SQL statement into a string from a form after a submit button is pressed and then run a query on the database with it. Having the knowledge of it is pointless with out the tool kit to make use of it.

Start here:
http://www.w3schools.com/php/default.asp
and here:
http://www.tizag.com/phpT/

dsf

Remember, I'm not a newbie programmer -- I'm a newbie web programmer who has already done some pretty significant coding -- just not anything with a web-based (or any, really -- remember, I program servers and drivers here) front-end and a database back-end.

Then you should know you need to get the syntax of a new language down so you can work with frameworks that are expansions of it.

Dragonmaster Lou

Remember, I'm not a newbie programmer -- I'm a newbie web programmer who has already done some pretty significant coding -- just not anything with a web-based (or any, really -- remember, I program servers and drivers here) front-end and a database back-end.

Then you should know you need to get the syntax of a new language down so you can work with frameworks that are expansions of it.

Oh, I agree -- it's just that the requirement of getting the syntax down is a no-brainer and I thought I had implied that I was going to do that (or already know the syntax, depending on the langauge I'm using) in my original post. What I really wanted to know is the "next steps" after getting the syntax down of whatever implementation language I plan to use, whether it's PHP, Python, JavaScript, INTERCAL, or whatever. Sorry if I wasn't clear. Oh, and thanks for those PHP links as well!

dsf

I personally hate PHP and would work exclusively in ASP if I could.

Anthony Heman

I personally hate PHP and would work exclusively in ASP if I could.

I've always seen the great programming language war as a list of different tools and people fighting about which tool is better. So to me, that's like saying you would always use a philips screwdriver instead of a flathead. They're not even that different in this case.

dsf

I personally hate PHP and would work exclusively in ASP if I could.

I've always seen the great programming language war as a list of different tools and people fighting about which tool is better. So to me, that's like saying you would always use a philips screwdriver instead of a flathead. They're not even that different in this case.

you misread or misinterpreted my statement. I use php all day, I just don't prefer it. That's not really starting a language war. I didn't say "Don't use PHP it sucks" It does not suck. Its pretty good at doing the job. I just don't like it because its open source and its functions are all over the place. But your analogy is off, you could say, "I like X tool maker over Y tool maker because X tools are more consistent and higher quality while Y tools are free".

dsf

as far as SQL injection and what not, there are some pretty good functions like "mysql_real_escape_string()" that have been built directly into PHP

Reference:
http://php.net/manual/en/function.mysql-real-escape-string.php

zehaeva

as far as SQL injection and what not, there are some pretty good functions like "mysql_real_escape_string()" that have been built directly into PHP

Parameterize your queries!

dsf

as far as SQL injection and what not, there are some pretty good functions like "mysql_real_escape_string()" that have been built directly into PHP

Parameterize your queries!

The parameterized query is considered better practice.

dsf

Here is another reason why I dislike PHP, I can't find a way to do parametrized without installing some extension. The ASP solution is so much nicer.

Dragonmaster Lou

Here is another reason why I dislike PHP, I can't find a way to do parametrized without installing some extension. The ASP solution is so much nicer.

PHP, best I can tell, is a total hack that ballooned into something much greater than it was ever meant to be used for. Personally, unless I'm modifying some pre-existing PHP code (Wordpress templates or something of that ilk), I think I'm going to stick with Python unless I have no other options. The way I see it, if I get good enough with the standard web programming techniques in Python, then it'll be mostly a case of learning the syntax to apply it to PHP or whatever (yeah, I know this is an oversimplification, but that's my general idea).

Apreche

PHP is a total hack that ballooned. However, do not discount it. I think of it like fingerpaint. It's really easy to make a big mess. It's really easy to get started without almost any learning whatsoever. Yet, it is still possible with great discipline and skill to make something great.

dsf

Here is another reason why I dislike PHP, I can't find a way to do parametrized without installing some extension. The ASP solution is so much nicer.

PHP, best I can tell, is a total hack that ballooned into something much greater than it was ever meant to be used for. Personally, unless I'm modifying some pre-existing PHP code (Wordpress templates or something of that ilk), I think I'm going to stick with Python unless I have no other options. The way I see it, if I get good enough with the standard web programming techniques in Python, then it'll be mostly a case of learning the syntax to apply it to PHP or whatever (yeah, I know this is an oversimplification, but that's my general idea).

I don't have the luxury of picking my language unfortunately it is imposed on me. Yes it is a TOTAL HACK! I could not agree with you more on this note.

Apreche

If you want to see how PHP can be done nicely lookup Yii, CakePHP or Symfony.

Dragonmaster Lou

PHP is a total hack that ballooned. However, do not discount it. I think of it like fingerpaint. It's really easy to make a big mess. It's really easy to get started without almost any learning whatsoever. Yet, it is still possible with great discipline and skill to make something great.

Oh, I agree. There is good stuff that has been written in PHP -- it's just not my first choice if I'm developing something from scratch. I figure I'll pick up PHP eventually, but one step at a time.