Jikan Update – September 2018

This update marks a new milestone for Jikan. I am super excited to announce quite a few things.

JikanPHP v2

JikanPHP has the backbone, parser of the Jikan REST service. It does all the parsing. With the help of some amazing contributors, we’ve rewritten JikanPHP. Making it more robust, professional and standard. V2 promises PSR2.

More on usage in the newly written Documentation which is powered by MkDocs – Material – providing a fresh look and better UX over the old documentation.

Lots of new stuff were added, lots of bugs were squashed, lots of effort was put into standard quality coding. There’s just too much to list here! Read more in the changelog.

Jikan REST Service

JikanREST has broke through an amazing mount of milestones. We’ve been getting over 3 million requests weekly! That’s no joke, behold the stats below.

Screenshot from 2018-08-30 05-47-51.png

That’s on average ~12,000 requests hourly, ~280,000 requests daily.

Service Improvements

Matsujo Hibiki, Jikan’s hosting sponsor has equipped the server with dual load balancers for fetching fresh requests and a master server that handles any cache. This means ZERO rate limiting from MyAnimeList and a 100% uptime service!

Jikan REST v3.0

I’ve extremely excited to announce JikanREST v3! Let’s go over a few of the many improvements.

Open Source

The entire codebase is open source and anyone can set up their own JikanREST instance!

No more Daily Limits!

That’s right! Daily limits of 5,000 requests per IP has been removed!

In return a much better throttling middleware has been introduced.

  • Cached requests are NOT throttled/counted against your limit
  • Clients can make upto 30 requests / minute
  • Clients can make upto 2 concurrent requests / second

That’s the limit over at api.jikan.moe. Obviously, if you’re hosting the instance, you can reconfigure these values, even remove them.

JikanREST v2.2 “Extended Requests” Depreciation Notice

In support of some things such as Jikan no longer utilizing ‘extended requests’ (due to performance issues and rather standing the endpoints as separate endpoints for JikanREST v3);

“Extended Requests” will depreciate on January, 1st 2019.

Developers are encouraged to start using api.jikan.moe/v3 ASAP.

That’s all for now folks.

Oh, we have a discord community set up! Come say hello!

Advertisements

Jikan News & Updates – Mid-2018

Okay, this news is almost a month old. Here goes.


Already 5 months into 2018 and I’ve already exciting news regarding Jikan. I wrote a post back in January – laying out the road map of Jikan for the current year. I had announced 4 more features that were to be done this year. I’ve completed 3 of them with User Related scraping to be done by the release of REST 2.3.

 

RELATED

 

Over the past year, Jikan has gained a huge traction, client and development wise. Here are the highlights of the past 6 months.

Jikan REST 2.2

With the release of REST 2.2, came many new features.

  1. More extended data for Anime and Manga (with the exception of reviews & recommendations – for now)
  2. Anime/Manga/People/Characters Search! This comes with advanced search filters and pagination support.
  3. Top Anime and Manga with advanced filters
  4. Season – To list the Anime airing this season and for other years/seasons.
  5. Schedule – Anime scheduling for the week for this season
  6. Meta – Experimental requests for getting usage stats for Jikan and most requested links by daily, weekly & monthly periods.

 

And some service changes.

  1. Jikan has moved domain to Jikan.moe. The previous (Jikan.me) domain has been discontinued.
  2. Jikan REST API is now being hosted in Tokyo (closer to MyAnimeList’s Tokyo server) by an awesome dude called Hibiki.

 

100% Jikan Open Source

That’s right. The entirety of Jikan has been open-sourced under MIT License. This includes the website, docs and REST API service.

This not only adds flexibility, but the code is easier to manage and deploy. There goes the days of patches having to wait till the next REST version. Now the RESTful services is updated as soon as a new JikanPHP version is out – this ofcourse will vary for major feature releases as I’ve to set up the controllers on the REST service.

 

Usage Stats

This is the Meta feature I mentioned.

 

It works by logging requests made in Redis and increasing the respective counters for that request. Here are some interesting usage links.

You can read more about the further usability.

 

Late 2018 Roadmap (REST 2.3)

So here’s a few stuff that will definitely be completed before the end of 2018. Perhaps in the upcoming months.

  • Top Characters/People
  • Anime/Manga Extended Data – Reviews & Recommendations
  • User Data – Profile, Watch History, Friends

 

Early 2019

This is given if the MyAnimeList’s new API hasn’t been publicly released yet or people haven’t started ditching Jikan.

  • JikanPHP (Core) – Rewrite. This will introduce JikanPHP 2.X.
    • Separation of the parser as an abstraction class for Requests & RegEx parsing
    • Faster Parsing – Rework Extended Requests.
  • Jikan REST 3.0 – Given the crazy amount of requests we’ve been gettings. The main problem is rate limiting from MyAnimeList since we’re making all these requests from one server, i.e one IP Address.
    • Rework Redis Database data caching
    • API Keys. Note: This won’t replace free, unmonitored GET requests. The current limit of 5,000 will be lowered down to encourage app/project developers to get an API key that will support higher rate limits.
    • Rework Extended Requests as separate API calls. This is a bottleneck right now as extended requests make 2 requests instead of one to merge the data for you into 1 request.
  • Relational data – Expand to other sites (maybe)

Jikan API – Vision 2018 🎆 [Unofficial MyAnimeList API]

So it’s 2018 and Jikan is now 1 year old! MyAnimeList announce late 2017 that they’ll be working on fixing up their API but until then I’ll have Jikan running around. I have some plans for Jikan that need to be done, hopefully by mid-2018 or earlier, depending on college.

 

READ

 

There are some things I’m still interesting in scraping off of MAL, here’s the list.

 

User Profile

Taking an example of my own profile;

 

There’s a lot of data available per user profile. The best part here would be their favorite characters, people, anime, manga and basic stats. The hardest part to extract here would be the user based “About Me” which is highly customizable. So this, I might consider parsing since MAL’s HTML source is already terrible enough.

 

Top Anime/Manga/People/Characters

These pages give you access to a paginated list of anime/manga/people/characters ranked by their popularity/favoritism by the community from #1 to the last ranking available. Tis a gold mine entry.

 

Anime/Manga/Person/Character Search!

The official MAL API already has this feature but it only returns the first page of results! It only allows simple string queries and requires user authentication for the API call to work, which is what Jikan is meant to over come. This has been a requested feature, so I’ll most likely be working on a parser for this in the months to come.

 

 

Extended Data for Anime/Manga

This has been in the prospect of Jikan since the beginning, but I’ve held off any other extended parsing other than characters/staff and episodes until recently as I begun making scrapers for Pictures, Videos & News related to the item. This trend will continue as there are more pages that consist of interesting data regarding an anime or manga. Especially the reviews page since this has the best data for sentient analysis and averaging of any show or manga.

 

Will be focusing on these 4 for this year! It takes time to mine pure data since scraping HTML off MAL means a lot of weird and round-about ways of doing things!

The first quality post ever

Last year, on the 3rd of April, I lay in excite as I bragged about a design update to whatever audience I had for my portfolio that went from 💩 to something that I would call an achievement that day.

fb

I went on about how content I was with it for the time being until about a week later the excite was replaced with a pit of angst within myself as I pondered on why everything looked so wrong.

Self-learning design has always been about observing it. Letting it sink in then having the option to replicate it. You have the tools. You know how to use them. But in the end, your mind is a blank slate.

 

And with that said, let’s get into a self-analysis of how I’ve taken it a step further over the year. Below you’ll witness a murder and the subjugator.

Now that you know, let us carry on with what was truly wrong with the former design. First of all.header-wat

I don’t know what came into mind when I designed this but as far as I can recall, it looked good on paper. What I simply wanted to implement was a Call To Action button that would scroll the viewer down to my projects section.

The button looks like a drop down and that pop out make no sense.

It was late 2016 when I caught up with simple, yet effective SVGs (in detail here). My conclusion was that abrupt ends towards the end of sections, were simply too terrible, especially the way I was executing it. SVGs came along and filled in that gap.

id-header-svg.png

With a much better looking CTA (but still not the best), I managed to compliment a ‘layered’ effect of different shades. Playing around with geometry transitions has always been a favorite.

Courtesy of my previous blog: https://bootyphpandi.wordpress.com/2015/10/27/the-beginning/

Unfortunately, I don’t have the files of my former design but those triangles slid in with a 45 degree rotation. The black and cyan-ish triangles you see there are simply squares that are hidden and partially shown on hover through that 45deg tilt. The black box tilted to the right where as the bottom, cyan box tilted upwards. This gave it this sleek look.

Back to the post.

There was one more shortcoming. (pun intended)

portfolio2

Siri courtesy: https://redd.it/5eht5c

The job of the CTA was to scroll the user down 100 pixels to the designated section…

It was not until later, I realized that this was terrible and put some distance between it by bring the ‘about’ section before the portfolio. Talk about proper hierarchy.

The cringe

What you see below that paragraph of cringe is a bunch of icons of the ‘skillsets’ I have. Before this, I had bars that represented this, similar to my 2015 design:

web4

But the thing is, there’s no limits to these skills. Something new comes out every day and I was simply pulling out the experience percentage based on my then beliefs.

Instead, I’ve replaced it with something generic and descriptive. This goes below the about section.skills.png

The simple “Biography”

Speaking of the about section, here’s how it looked before.

about-old

Talk about a cringeful, long description with font size big enough for sufficient for the elders only.

I’ve changed this entirely. It’s now the first section, so it’s right below the header.header

Pretty dank, aye?

Navigation, ahoy

Before we progress, there is one more thing we need to talk about and that’s the sticky navigation bar that follows you down.

Before I had this dull piece of stick nav-old

And now, there’s thisnav-new

The portfolio’s new design is supposed to have more contrast. As you may have noticed (or not), the height of the sticky nav bar is much smaller now. It adds more breathing space to the page by tens of pixels.

There’s also a little border at the bottom to make it ‘pop-out’ with a much more elaborate shadow at the bottom than the former.

Relevant Oatmeal

make it pop!

Work of ‘The Oatmeal’ (http://theoatmeal.com/comics/design_hell)

 

The Portfolio

Let’s talk about the portfolio section. Before it was boring cards with direct links to the download or view button and the cards themselves had some design issues. I don’t have a pre-existing screenshot but they looked like the ones now except the images got squeezed or stretched.

portfolio.png

I’ve updated that part and divided the portfolio into 3 categories, Client, Designs & Apps.

Taking guess would be enough to realize that the client category is for client work, the design category is for web designs I make myself, be it free or premium. And Apps are app websites I’ve developed.

But that’s not the best part of the portfolio, you may see a familiar design on hover here lacking the download and view buttons. That’s because it’s begging to be clicked.portfolio-hoverIf it’s downloadable, a small download button appears as well. Now once you click on the card, it fetches the details for the item via AJAX and using Bootstrap Modals, we get this beauty.

portfolio-modal.png

portfolio-modal2.png

I can safely say that this is probably the first modal I’ve designed, hence some design issues here too. I need to rework the bottom part, not sure at the moment how though. But this is what I’ve got. It’s quick and simple. Bootstrap Modals are a bonus with User Experience. Click on the X or behind the modal and you’re back to the portfolio.

 

The Bottom

The contact section may look nearly identical to the former but rest assured it’s been revamped from a user experience perspective. Instead of the good old page reload for the submission process, it now utilizes the power of AJAX for a synchronous update on the spot. It’s acquainted with Sweet Alerts and Google’s reCAPTCHA which provides some mercy on the database.

There’s 860 messages of which 4 are actual messages and the rest of it are spam. My website is quite popular with the bots – heh.

ezgif.com-resize.gif

Messages from Bots

Nevertheless, here’s how fast and simple it is now for anyone to drop in a message.

contact gif

As for the footer, I’ve removed the bulky useless section it had before and replaced it with something simple.

footer.png

You may have realized, I removed the Twitter Feed! Actually, I had plans for it. Right now I’m using free hosting from 000webhost. It disallows the use of REST APIs and that sucks. I actually have the whole Twitter API, cache, etc ready to roll out. All it’s lacking is the design. I thought I’d put it in the footer but that’s just meh. I thought I should promote my blog posts on my portfolio as well so I’ve been thinking of making an entire new section dedicated to a twitter feed (“recent rambling”) and blog posts (“recent posts”). I haven’t designed anything yet as they both use REST APIs and until I move to a proper limitless domain, I won’t be updating on this.

 

Enough of the design, Let’s talk about the inside.

It’s what’s on the inside that counts.

I’ve re-coded everything in PHP, still not following the MVC structure but rather my own structure, which is truly odd to explain.

I’ve moved from using CSS to LESS. LESS is basically better CSS syntax. Next up, I’ve started using Bootstrap as the front end framework. As much as I promised to use only made-from-scratch stuff, I’ve really been slowing myself down. Bootstrap has built in modals, grids, etc which really put off workload. Both of these combined proved a faster and easier workflow. I’ve really cut in half the time it takes to code a design.

 

What now?

I’m still not content with how it looks. I’ve still yet to implement the section that consists of my blog and twitter posts. Also, the website lacks responsiveness (not adapted for mobile or tablets). I’m too lazy to add it now but I’ll probably ninja add it later on.

One more thing!

Branding. So far I’ve not used any logos to represent myself and really needed a favicon (that tiny icon you see on your tab before the web page’s title). So I followed my life motto, “Why not?”, and thus utilized my expert GIMP skills (2poor4photoshop) and came up with the following.

logo.png

 

As you can see, it looks terrible at the edges, but that’s not going to show in a small logo or favicon and I’m too lazy to perfect the small details it at the moment. Maybe later on?

This concludes the berating of my former portfolio and what I did to upgrade it.lolwut

ayylmao

 

A refreshing look using the same palette

So, I’ve redesigned the portfolio. Sort of. Even re-coded the backhand so it’s OOP that makes actual sense. That’s the gist of it.

SVG

It’s been used around a lot since the past year and I thought “why not?”. I made some really simple SVGs that are right-angled triangles to give the edges of containers a padded and – er – better look? THEY LOOK GREAT, and that’s what I believe matters? Okay. In addition, these are vector graphics and are ludicrously small in size. Going to use them more in further projects after I get a good grip on them.

design2

BOOTSTRAP

The only reason I’ve started using bootstrap is because of its preset configuration, glyphs and the responsive grids. Oh, the responsive grids – never have I ever made something without spending much time on the frames.

design3

 

I pat my self for making it look better but I still believe it’s got it’s ways to go before it reaches perfection. The thought of upgrading the current design rather than making a complete new one was really proper choice.

 

WHAT’S NEXT?

I’m planning a twitter feed and a cool little vertical ‘timeline’ under my bio which would show which programming language/feat I achieved at which year.

I also think that the SVG triangles are a little to large? Not sure, but I’m going to play around with that.

The Official Follow Up

So, I’m back with another blog. I do realise that the previous one (bootyphpandi.wordpress.com) was lacking a decent name and so I took it upon myself (again) to bring it to a professional state. I had plans of making my own CMS but again I realised that I’d have to hit up AngularJs and some more alpha type stuff to make it look like a decent CMS. Plus due to the lack of time, I’ll be using this as my official blog.

Shoutout to Tonal theme as I really love this minimalistic freebie.

Some Updates

Portfolio Polishing

I updated my own portfolio (irfandahir.com). The design was left unfinished so I polished it out a bit after recieving some insight and critique from forum boards. I still feel it’s lacking so I’m devising plans to make it look nicer.

 

Omilos by  id

Introducing Omilos

I’ve made another freebie, Omilos. I thought “Omilos” was greek for “something big” but my greek buddy corrected me once more. Nevertheless, the design is upto the level of being used. IMO it is lacking some design fundamentals and has some flaws but it will get your job done as it’s coded as cleanly possible. If you hire a designer or have some coding skills, you’ll be able to adjust it to your needs.

Demo | Download

 

Project Extract Cloud

Project.Extract 4

If you’re a CS2D player then you might know what this is, if not then here’s a breif explaination. Project.Extract (including legacy versions 1, 2 & 3) have been downloadable apps which run through your browser with the dependacy of Apache & PHP (WAMP, LAMP). CS2D generates a fair amount of logs files and so I created a PHP Library which would extract useful amounts of information from these logs. And Project.Extract is the visual version of that. The Legacy versions 1-3 only extracted user information and had text based searching.

So a year later, after leveling up multiple times in PHP I realised I could extract so much more. I’ve developed a PHP Library, Log Miner, for it which acts as a core for Project.Extract 4. Both the PHP Library and PE4 are in works. The difference between legacy versions and this is that this has the capability to extract ALOT more from your logs. Every single detail. And the awesome part? It’s both a web based app if you don’t know how to set it up and downloadable which removes limits. I’ll talk more about it once I’m ready to deploy it.

If you’re interested then these are repos you should keep an eye on.

[REPO][PHP Library] Log Miner

[REPO] Project.Extract 4

 

That’s it for now.