The easiest way to scrape details from a myspace profile page with php (you won’t believe how simple it is)

It’s amazing how just a little optimization on the part of myspace makes crawling their site so much easier. We’re going to scrape the user detail (name, age, sex, etc..) from a profile, using the header info like so:

Set your myspace url:

$your_profile_url = ‘http://www.myspace.com/waxjelly’;

Now grab the file using the “file()” function. (We want an array, so we can crawl it and use “trim()” to clean it up)

$file = file($your_profile_url);

Now we setup a string and loop through the profile array to clean it up.

$profile = ”;
for ($i=0; $i<count($file); $i++) {
$profile .= trim($file[$i]);
}

Now we use simple explode functions to do the rest of the work. What we’re looking for is the “<meta” tag at the beginning of the file that will grab the basic details. (This got thrown in place when myspace partnered with google for search optimization. Thanks, guys.).

$det_arr = explode(’<meta name=”description” content=”myspace profile - ‘, strtolower($profile));
$det_arr = explode(’” />’, $det_arr[1]);

We’ve got the whole string that we need, but now we need to separate it into an array we can manage. This is the prettiest part of this script. Myspace prints the element even if it’s blank, so if you leave your city blank, we’ll get a nice little “…Male, , Texas,…” string. Note the double commas. That means we can explode on the comma, and still get a consistently indexed array. (Index 3 will always be city, even if it’s blank. And index 4 will always be state, even if city is blank. Make sense?)

$details = explode(’,', $det_arr[0]);

Now that we have the array that we want, we simply assign them to a more usable system.

$det['name'] = $details[0];
$det['age'] = $details[1];
$det['sex'] = $details[2];
$det['city'] = $details[3];
$det['state'] = $details[4];
$det['country'] = $details[5];
$det['phrase'] = $details[6];

… and print the results.

print_r($det);

That’s it! You can get a working version of the script here. Enjoy!

Your Friend and Mine,
Meshach

[digg=http://digg.com/programming/The_easiest_way_to_scrape_details_from_a_myspace_profile_page_with_php]

CSS tricks you may not know… (pt 1)

Just found this from webcredible. It lists 10 “hacks”. I found a few of them a little … obvious… so I wouldn’t call them all hacks, but here are a few that I find myself using often.

ONE: “When styling fonts with CSS you may be doing this:

font-weight: bold;
font-style: italic;
font-variant: small-caps;
font-size: 1em;
line-height: 1.5em;
font-family: verdana,sans-serif;

There’s no need though as you can use this CSS shorthand property:

font: bold italic small-caps 1em/1.5em verdana,sans-serif;

Much better! Just a few of words of warning: This CSS shorthand version will only work if you’re specifying both the font-size and the font-family. The font-family command must always be at the very end of this shorthand command, and font-size must come directly before this. Also, if you don’t specify the font-weight, font-style, or font-variant then these values will automatically default to a value of normal, so do bear this in mind too.”

TWO: “The box model hack2 is used to fix a rendering problem in pre-IE 6 browsers on PC, where by the border and padding are included in the width of an element, as opposed to added on. For example, when specifying the dimensions of a container you might use the following CSS rule:

#box {
width: 100px;
border: 5px;
padding: 20px;
}

This CSS rule would be applied to:

<div id=”box”>…</div>

This means that the total width of the box is 150px (100px width + two 5px borders + two 20px paddings) in all browsers except pre-IE 6 versions on PC. In these browsers the total width would be just 100px, with the padding and border widths being incorporated into this width. The box model hack can be used to fix this, but this can get really messy.

A simple alternative is to use this CSS:

#box {
width: 150px;
}

#box div {
border: 5px;
padding: 20px;
}

And the new HTML would be:

<div id=”box”> <div>… </div> </div>

Perfect! Now the box width will always be 150px, regardless of the browser!”

I’m sure there will be more to come, but these two are a great place to start with making better CSS-based sites.
Your Friend and Mine,
Meshach

How to write a custom Smarty function in php

In the interim between posts at waxjelly, we’ve collectively become addicted to Smarty. In addition to this, we’ve become addicted to extending and customizing smarty to do a bunch of crap that we find ourselves repeating, and essentially building our own custom framework on this humble-for-no-good-reason templating system. In this tutorial, we’ll write our first custom smarty, and get the ball rolling in the right direction for being able to do pretty much anything you want to do with … okay, this post is going to say “Smarty” a lot, in case you can’t already tell. Let’s get started.

  1. Locate your smarty plugins folder (mine is outside the document root, in the smarty/plugins directory)
  2. Create a new file and name it “function.waxjelly_hello.php”
  3. Create another file and name it “hello.tpl”
  4. Create a third file and name it “Index.php”

We’ll start by writing the index.php file, which will do the following:

  1. include the smarty class file
  2. instantiate an object of the smarty class
  3. set the path to our templates directory
  4. set the path to our compile directory
  5. set a smarty variable assignment
  6. display our class

Next, we’ll write our template file, which will do the following:

  1. call our custom smarty function
  2. pass it a static attribute
  3. pass it a smarty variable

Finally, we’ll write our custom smarty function, which will do the following:

  1. receive the parameters sent to it
  2. receive the global smarty object
  3. send an error to the smarty object if the required parameters are not set
  4. display the values of all received parameters in a basic format

Now that we’ve got that down, here are the files associated with this tutorial. Feel free to download, open up, and follow along, but keep in mind that there is some configuration in order to get it to work, which will all be covered in this post.

  1. index.php

    First, we include the smarty class file…

    include(PATH_TO_YOUR_SMARTY_FOLDER . ’smarty/Smarty.class.php’);

    Next, we instantiate the smarty object…

    $smarty = new Smarty();

    Now reset the template directory path…

    $smarty->templates_dir = PATH_TO_YOUR_TEMPLATES_FOLDER . ‘templates/’;

    … and reset the compile directory path…

    $smarty->compile_dir = PATH_TO_YOUR_COMPILE_FOLDER . ‘templates_c/’;

    Now that that’s done, we assign the variable that will be needed for this page…

    $smarty->assign(’name’, ‘Meshach’);

    Finally, we’re going to display the template file…

    $smarty->display(’index.tpl’);

  2. index.tpl

    In this file, we’re going to call the custom function, and pass it values for “greeting” and “name”.
    NOTE: the “name” attribute is being filled by the assigned variable from the index.php, and the greeting attribute is being hard-coded

    {waxjelly_hello greeting=’Hello’ name=$name}

  3. function.waxjelly_hello.php

    When we’re writing our custom smarty function, it’s name has to start with “smarty_function” and end with “_your_function_name“. All of the attributes are being passed into one array, which we’re calling “params”. We also include access to the parent smarty object with the nifty “&$smarty” var, which will let us pass it values for error messages without breaking the page load.

    function smarty_function_waxjelly_hello ($params, &$smarty) {

    This line of code makes “greeting” a required attribute, and sends an error through the smarty object if it’s not set…

    if (empty($params['greeting'])) {
    $smarty->_trigger_fatal_error(”[waxjelly_hello] param ‘greeting’ cannot be empty “);
    return;
    }

    …since we’re not requiring “name” as an attribute, we’re setting a default value for it if it’s not set

    if (isset($params['name'])) {
    $name = $params['name'];
    } else {
    $name = ‘(whoever you are).’;
    }

    … now we build a string to return to the page calling this function

    $return = ‘<div class=”myclass”>’;
    $return .= $params['greeting'] . ‘, ‘ . $name;
    $return .= ‘</div>’;

    Finally, we return the string, which will be compiled and echo’d

    return $return;

    }

That’s it! You’ve just written your first smarty plugin. We’ll be back with some more complex ones, but for now, enjoy cracking these things out to help your template-based site development. If you wanna get some ideas and inspiration (and very usable code in most cases), check out the smarty plugin directory.

Your Friend and Mine,
- Meshach

[digg=http://digg.com/programming/How_to_Write_a_Custom_Smarty_Function_in_PHP]

Why does myspace CSS work the way it does? Pt 2

If you haven't read our post on this actual code, you can view Part 1 of this series here.

First, let's cover the basics of how CSS works. When you see the "" tag, that means that everything between the two are overriding the default html tag's display features. For example, w3schools has a set of default style values such as font-weight: bold, and font-size: 20pt (or so). In order to override that in css, we'd do something like this:

HTML:
  1. h1 { font-weight: normal; font-size: 12pt;}

Note that we start by declaring a new style "". Then, we declare which tag we want to change "h1", followed by an open curly bracket "{", a list of style settings, then the closing curly bracket "}" implying the end of our new styles.

Now our style sheet has overridden the default value for h1 tags throughout the document, and will be reflected everywhere you have a

HTML:
  1. <h1>sometext</h1>

...tag set. Now, we can create "classes" and "id's" in css that allow us to use custom styles. For example, myspace uses a custom css class which they titled "contactTable". In the html, they represent this with the following tag:

HTML:
  1. <i><span class="contactTable">"... some crappy images ..."</span></i>

In our myspace style code above, you'll see something like the following code:

HTML:
  1. <i><style type="text/css"><br /> .contactTable {display:none; height: 0 !important; visibility:hidden; }

Note the "." before the "contactTable" title. This tells the css to look for contactTable as a "class", as opposed to a custom "id", or a built-in CSS tag. Also, you'll notice that what we're doing with this tag is essentially just squashing it into nothing. That's because the purpose of this code is to HIDE everything. There are TONS of code samples out there to show you how to change the background, colors, etc. And if we have time, we'll add some of that here, but for now, we're just "turning it off".

This is a great time to interject that it would be HUGELY beneficial to both the users of myspace AND myspace themselves for THEM to give you the option of turning off sections of data like this, as it could GREATLY reduce bandwidth on their part. However, they're dumb, and you're not, so let's continue to give you more responsibility and control. That's why you're here, right?

Now that you're a CSS master (it's really that simple), let's look at this code again and break it down line-by-line.

This first section of code turns off ALL FLASH CONTENT.

HTML:
  1. embed, object {display:none;}</style></i>

So, if you do end up with some div overlay layout, and you want to add say, a youtube video embed in your profile, just add a style attribute to the object tag, like so (added code in bold):

HTML:
  1. <i><span class="mceItemObject"  type="application/x-shockwave-flash" data="http://www.youtube.com/v/TTV0Aa4lC04" height="350" width="425"<b> style="display: inline;"></span></i> <span name="movie" value="http://www.youtube.com/v/TTV0Aa4lC04" class="mceItemParam"></span> <span name="allownetworking" value="internal" class="mceItemParam"></span> <span name="allowScriptAccess" value="never" class="mceItemParam"></span> <span name="enableJSURL" value="false" class="mceItemParam"></span> <span name="enableHREF" value="false" class="mceItemParam"></span> <span name="saveEmbedTags" value="true" class="mceItemParam"></span> <span name="wmode" value="transparent" class="mceItemParam"></span>

Now, back to the code. Next, we turn off the myspace-generated custom tags. This way, we have direct access to every instance of the use of any of these tags, and we don't have to "target" them, which I'll explain in a second. So, in order to save ourselves from writing the same code over and over again for every class we want to turn off, we simply list them, separated by commas, and then apply a style, like so:

HTML:
  1. <i>.contacttable, .whitetext12, .nametext, .lightbluetext8, .orangetext15, .blacktext12,.btext, .redtext, .redbtext{ display:none; height: 0 !important; visibility:hidden; }</i>

If we want to "target" an instance of a tag, say for example that we want to only change an h1 tag if it appears in a nest of other tags like so,

...then we would address it in css like so:

HTML:
  1. <i>table tr td div h1 { ... some styles }</i>

You'll notice that we don't have any commas separating our list this time, which tells CSS to "target" the h1 tag when it's nested in a div, which is nested in a table details, which is nested in a table row, which is nested in a table.

NOTE: "nesting" means that tags are opened, then other tags are opened, then the second tag is closed, then the first tag is closed. Like this:

HTML:
  1. <parent><child></child></parent>

In this example, the "Child" tag is "nested" inside of the "Parent" tag. Make sense? Good. Glad you're following.

Having said that, you'll see LOTS of targeting in all the millions of bits of myspace code around the myspace'o'sphere. Don't be intimidated. It's a result of a VERY badly designed site (myspace), and the sheer power of a well-written code base (CSS), blended with the determination of a creative user (You). Fighting the system, and getting what YOU want, you continue your code with the following:

... when td's are nested 4 deep...SUCH AS:

HTML:
  1. td td td td { border:0px; width:0px; text-align:left; }

... all table and td tags... SUCH AS:

HTML:

HTML:
  1. table, td { padding:0px;width:;background-color:transparent}

... when tables are nested 3 deep... SUCH AS:

HTML:
  1. <table>...</table><table></table><table>..."

HTML:
  1. table table table { padding:1px; height:.01%; width:100%;}

... tables nested 2 deep, tables nested 4 deep, table, tr, and td tags... SUCH AS:

HTML:
  1. </table><table>...</table><table>" AND "</table><table>... </table><table>...</table><table>...</table><table>..." AND "</table><table>" AND "<tr>" AND " <td>"

HTML:
  1. <i> table table, table table table table, table, tr, td {height:0px;!important;border:0px;!important}</i>

... tags that have "class=text" in them, when tags are nested in font tags which are nested in div tags which are nested in a table, when a font tag is nested in any tag containing a "class=navbar" attribute, when a font tag is nested in a td which is nested in a tr ...SUCH AS:

HTML:
  1. <a class=text>" AND "<table>... <div>... <font>... <a>..." AND "<table>... <div>... <div>..." AND "... class=navbar>... <font>" AND "<tr>... <td>... <font>..."

HTML:
  1. <i> a.text, table div font a, table div div, .navbar font, tr td font { visibility:hidden; display:none; height:0px;!important;}</i>

... tables nested 4 deep, td's with "class=text" nested in a table nested 4 deep, tables nested inside of 2 nested td's which each contain "class=text" attributes ...SUCH AS:

HTML:
  1. <table>...</table><table>...</table><table>...</table><table>... " AND "</table><table>...</table><table>...</table><table>...</table><table>...<td class=text>" AND "</td><td class=text>...</td><td class=text>...<table>"

HTML:
  1. <i> table table table table,table table table table td.text, td.text td.text table { display:none;}</i>

... table tags nested 2 deep inside of td tags with "class=text" attributes ...SUCH AS:

HTML:
  1. <td class=text>...<table>...</table><table>..."

HTML:
  1. td.text table table { display:inline; visibility:visible;}

... table tags nested in td tags with "class=text" attributes nested in a tr tag which is nested in a table tag which is tested in a td which is nested in a table ...
SUCH AS:

HTML:
  1. </table><table>... <td>...<table>...<tr>...<td class=text>...<table>..."

HTML:
  1. table td table tr td.text table { visibility:hidden;}

... tables nested 2 deep inside td tags with "class=text" nested in tr tag which is nested in a table which is nested in a td which is nested in a table ...SUCH AS:

HTML:
  1. </table><table>... <td>...<table>...<tr>...<td class=text>...<table>...</table><table>..." AND "</table><table>... <td>...<table>...<tr>...<td class=text>...<table>...</table><table>...<td class=text>..."

HTML:
  1. table td table tr td.text table table, table td table tr td.text table table td.text { visibility:visible;}

And of course, we close the style...
[html][/html]

Now that you're a CSS guru, go and hack the hell outta your myspace page. I'm sure we'll be posting again soon on how to do even more. For now, enjoy.

Your Friend and Mine,
Meshach

256 Comments

Hide everything on your myspace profile (including music player!!!) Pt 1

***************************************************************
ADDENDUM: I didn't think about this before, but it's been brought to my attention that some of you may just flat out want to turn off your profile. I have to warn you, that if myspace finds your profile with ads turned off, they'll cancel your account. It's free, so you can always sign up again, but it's a real pain, and you will likely not get your user name back. However, if you so desire, it's one line of css, and that line is:

<style type=text/css>table, tr, td, embed, object {display: none;}</style>

Now, back to the regular stuff...
***************************************************************

This doesn't create a div overlay, it' just turns everything off so you can do the work designing the whole page yourself. njoi!

Copy the following code starting here --->>

<style type=text/css>

embed, object {
   display:none;
}

.contacttable, .whitetext12, .nametext, .lightbluetext8, .orangetext15, .blacktext12,.btext, .redtext, .redbtext {
   display:none;
   height: 0 !important;
   visibility:hidden;
}

td td td td {
   border:0px;width:0px;text-align:left;
}

table,td {
   padding:0px;
   width:0;
   background-color:transparent;
}

table table table {
   padding:1px;
   height:.01%;
   width:100%;
}

table table,table table table table,table,tr,td {
   height:0px !important;
   border:0px !important;
}

a.text,table div font a,table div div,.navbar font,tr td font {
   visibility:hidden;
   display:none;
   height:0px !important;
}

table table table table,table table table table td.text, td.text td.text table {
   display:none;
}

td.text table table {
   display:inline;
   visibility:visible;
}

table td table tr td.text table {
   visibility:hidden;
}

table td table tr td.text table table,table td table tr td.text table table td.text {
   visibility:visible;
}

</style>

<<---- And ending here.

Great, now why the hell does that work? Let's try and break it down.

Your friend and mine,
-Meshach

[digg=http://digg.com/programming/Hide_everything_on_your_myspace_profile]

274 Comments

Simple YouTube API Class in PHP: Redeaux

Good evening, people of the corn. I am absolutely certain that the grand majority of you have been waiting in the wings for the latest installment of WaxJelly's now famous YouTube API class. Thanks for waiting. In the wings. Wherever they are.

We're starting off slow again on this one, so I'll warn you, all the functionality may not quite be there yet, but this class is destined to continue to improve and become more simple to implement and use. Howeve, at the moment, using this thing is about as easy as falling off a bike.

Let's get started. First, let's talk about what you'll need in order to implement this class.

  • A YouTube API Developer's Key
  • A server running PHP5
  • 20 minutes or so
  • Popcorn
  • Red Hair
  • I could go on, but these stopped being true at bullet point #3

This is an example of what your end result should look like:

WaxJelly YouTube API Class

Now, let's talk about the code to actually make this class work for you. You'll find the following code in the index.php file included in the downloadable package at the end of this post.

Include the youtube class and instantiate it with the object $yt

PHP:
  1. include('youtube.class.php');
  2. $yt = new youtube('YOUR_API_KEY');

...set the amount of results per page, in this case, some arbitrary number like 19 should work. The default is 25, so if you don't set this at all, that's what you'll get

PHP:
  1. $yt->set('per_page', 19);

...get videos by a tag (default search string is "mutemath" in this case ,you can change it to whatever you want.

PHP:
  1. $res = $yt->videos('mutemath');

...build the video list array

PHP:
  1. $video_list = $res['video_list']['video'];

Note that this class sets a whole buncha CONSTANTS that you can use in your script to display lots of data. Here's a list of the constants that are available:

  • YOUTUBE_PREV_PAGE = the current page -1
  • YOUTUBE_NEXT_PAGE = the current page +1
  • YOUTUBE_PAGE = the current page
  • YOUTUBE_TOTAL_PAGES = total pages in the search results
  • YOUTUBE_VIDEO_ID = the currently selected video's unique ID (for display purposes, or in the future, to call youtube's get_details method an retrieve further information, like actual comments by users on this video)
  • YOUTUBE_VIDEO_INDEX = the currently selected video's index in the video_list array (to get all the data from the array)

Later in the file, we loop through our $video_list array and format to display the videos. We do this like so:

...SETUP PAGINATION TO DISPLAY THE NEXT AND PREVIOUS PAGES OF RESULTS, AS WELL AS THE CURRENT PAGE, AND THE TOTAL NUMBER OF PAGES

PHP:
  1. <div class="paginate">
  2. <a href="?page=<?=YOUTUBE_PREV_PAGE ?>"><<prev</a>
  3. :: page <?=YOUTUBE_PAGE ?> of <?=YOUTUBE_TOTAL_PAGES ?> ::
  4. </a><a href="?page=<?=YOUTUBE_NEXT_PAGE ?>">next>></a>
  5. </div>

...BUILD A TABLE OF RESULTS, LOOPING THROUGH 3 COLUMNS ON EACH ROW

HTML:

...loop through each video in the list and display it for design purposes

PHP:
  1. $i = 0;
  2. foreach ($video_list as $k => $v) {

...limit the title's length so it doesn't break the design

PHP:
  1. $title = substr($v['title'], 0, 15) . '...';
  2. echo "<td class='thumb'>
  3. <a href='?video_id={$v['id']}'><img src='{$v['thumbnail_url']}'/></a><br />
  4. <span class='title_short'>{$title}</span><br />
  5. <span class='length_short'>{$v['length_seconds']}</span>
  6. <span class='rating_short'>{$v['rating_avg']} / {$v['rating_count']}</span>
  7. </td>";
  8. $i++;

...only 3 videos per row

PHP:
  1. if ($i == 3) {
  2. $i = 0;
  3. echo '</tr><tr>';
  4. }
  5. }
  6. ?>
  7. </tr>
  8. </table>

Finally, we setup the display on another part of the page to actually play a video we've selected, and display the rest of the data we've gotten on that video (without using the get_details method from the api)

...first, we setup the title of the current video.

PHP:
  1. <h2>
  2. <?=$video_list[YOUTUBE_VIDEO_INDEX]['title'] ?>
  3. </h2>

Next, we setup the flash object call from the standard youtube format, and pass it the value for the video id

PHP:
  1. <object width="425" height="350">
  2. <param name="movie" value="http://www.youtube.com/v/<?=YOUTUBE_VIDEO_ID ?>"></param>
  3. <param name="wmode" value="transparent"></param>
  4. <embed src="http://www.youtube.com/v/<?=YOUTUBE_VIDEO_ID ?>" type="application/x-shockwave-flash" wmode="transparent" width="425" height="350"></embed>
  5. </object>

We can also display all the extra information, like video description, and the total number of views, comments, and the list of tags (all of which are passed to you through our youtube class in the current index of the $video_list array

PHP:
  1. <div id="description">
  2. <?=$video_list[YOUTUBE_VIDEO_INDEX]['description'] ?>
  3. </div>
  4. <div class="stats">Views: <span class="statVal"><?=$video_list[YOUTUBE_VIDEO_INDEX]['view_count'] ?></span><br />
  5. Comments: <span class="statVal"><?=$video_list[YOUTUBE_VIDEO_INDEX]['comment_count'] ?></span><br />
  6. Tags: <span class="statVal"><?=$yt->tags_for_video($video_list[YOUTUBE_VIDEO_INDEX]['tags']) ?></span><br />
  7. </div>

Download the source files here: ZIP | RAR
View a working version HERE

Peace love and peace.
[digg=http://digg.com/programming/WaxJelly_YouTube_API_Class_Redeaux]