Jikan Update – October 2018

Stats for JikanREST have stayed pretty much the same for Jikan in the past month, in the range of 2-3 million requests per week. I don’t think that’d be dying down anytime soon.Screenshot from 2018-09-30 04-48-58.png

This is rather a minor update so I’ll keep to the point.

JikanPHP v2.0.0 Stable

After months of being stuck in RC, we’ve plowed through the field of bugs and finally released v2.0.0 Stable a few days back. Rejoice!

v2.1.0 Stable is nearly ready for release as well which features new constant updates, bug fixes, and User List Parsing.

JikanREST v3.1 Release

JikanREST runs on the dev-master branch rather than any published released so it’s always upto date with the recent additions to the parser.

I’ve added a few new features to the REST service.

1. Nullable Date Range Props

A terrible problem with MyAnimeList is that when it returns date ranges for Anime & Manga, it’s usually data submitted through the community users and sometimes doesn’t pass the validation checks as MAL Editors/Mods are going through them. This brings up very inconsistent Date formats. Which sometimes break the parser.

Therefore as per a discussion with contributors, we worked on and introduced Nullable Dates. Which look like this:

Screenshot from 2018-09-30 04-59-53.png

Quite convenient, if you ask me.

 

2. User Lists

A much awaited feature! There’s this internal MAL API which returns data in JSON. The problem being that it does not have CORS support and one would have to rather rely on a CORS Proxy which can bring down the response time. Thus, with a bit of standardization of the schema, I’m introducing User Lists to Jikan.

Usage Info

 

3. Season Archive

MAL has a season archive page where it lists a valid amount of Years and Seasons that it currently has in it’s database. This feature was complete when JikanREST 3.0 was released but not included. It has now been added to the REST service.

Usage Info

 

Going Forward

There’s still quite a lot of features that are included in the Roadmap and I wish to add. Unfortunately will be getting busy now due to IRL stuff, but try to will keep consistent updates to Jikan. PRs are much welcome!

Jikan Roadmap

Advertisements

Jikan Update – September 2018

This update marks a new milestone for Jikan. I am super excited to announce quite a few things.

JikanPHP v2

JikanPHP has the backbone, parser of the Jikan REST service. It does all the parsing. With the help of some amazing contributors, we’ve rewritten JikanPHP. Making it more robust, professional and standard. V2 promises PSR2.

More on usage in the newly written Documentation which is powered by MkDocs – Material – providing a fresh look and better UX over the old documentation.

Lots of new stuff were added, lots of bugs were squashed, lots of effort was put into standard quality coding. There’s just too much to list here! Read more in the changelog.

Jikan REST Service

JikanREST has broke through an amazing mount of milestones. We’ve been getting over 3 million requests weekly! That’s no joke, behold the stats below.

Screenshot from 2018-08-30 05-47-51.png

That’s on average ~12,000 requests hourly, ~280,000 requests daily.

Service Improvements

Matsujo Hibiki, Jikan’s hosting sponsor has equipped the server with dual load balancers for fetching fresh requests and a master server that handles any cache. This means ZERO rate limiting from MyAnimeList and a 100% uptime service!

Jikan REST v3.0

I’ve extremely excited to announce JikanREST v3! Let’s go over a few of the many improvements.

Open Source

The entire codebase is open source and anyone can set up their own JikanREST instance!

No more Daily Limits!

That’s right! Daily limits of 5,000 requests per IP has been removed!

In return a much better throttling middleware has been introduced.

  • Cached requests are NOT throttled/counted against your limit
  • Clients can make upto 30 requests / minute
  • Clients can make upto 2 concurrent requests / second

That’s the limit over at api.jikan.moe. Obviously, if you’re hosting the instance, you can reconfigure these values, even remove them.

JikanREST v2.2 “Extended Requests” Depreciation Notice

In support of some things such as Jikan no longer utilizing ‘extended requests’ (due to performance issues and rather standing the endpoints as separate endpoints for JikanREST v3);

“Extended Requests” will depreciate on January, 1st 2019.

Developers are encouraged to start using api.jikan.moe/v3 ASAP.

That’s all for now folks.

Oh, we have a discord community set up! Come say hello!

Jikan News & Updates – Mid-2018

Okay, this news is almost a month old. Here goes.


Already 5 months into 2018 and I’ve already exciting news regarding Jikan. I wrote a post back in January – laying out the road map of Jikan for the current year. I had announced 4 more features that were to be done this year. I’ve completed 3 of them with User Related scraping to be done by the release of REST 2.3.

 

RELATED

 

Over the past year, Jikan has gained a huge traction, client and development wise. Here are the highlights of the past 6 months.

Jikan REST 2.2

With the release of REST 2.2, came many new features.

  1. More extended data for Anime and Manga (with the exception of reviews & recommendations – for now)
  2. Anime/Manga/People/Characters Search! This comes with advanced search filters and pagination support.
  3. Top Anime and Manga with advanced filters
  4. Season – To list the Anime airing this season and for other years/seasons.
  5. Schedule – Anime scheduling for the week for this season
  6. Meta – Experimental requests for getting usage stats for Jikan and most requested links by daily, weekly & monthly periods.

 

And some service changes.

  1. Jikan has moved domain to Jikan.moe. The previous (Jikan.me) domain has been discontinued.
  2. Jikan REST API is now being hosted in Tokyo (closer to MyAnimeList’s Tokyo server) by an awesome dude called Hibiki.

 

100% Jikan Open Source

That’s right. The entirety of Jikan has been open-sourced under MIT License. This includes the website, docs and REST API service.

This not only adds flexibility, but the code is easier to manage and deploy. There goes the days of patches having to wait till the next REST version. Now the RESTful services is updated as soon as a new JikanPHP version is out – this ofcourse will vary for major feature releases as I’ve to set up the controllers on the REST service.

 

Usage Stats

This is the Meta feature I mentioned.

 

It works by logging requests made in Redis and increasing the respective counters for that request. Here are some interesting usage links.

You can read more about the further usability.

 

Late 2018 Roadmap (REST 2.3)

So here’s a few stuff that will definitely be completed before the end of 2018. Perhaps in the upcoming months.

  • Top Characters/People
  • Anime/Manga Extended Data – Reviews & Recommendations
  • User Data – Profile, Watch History, Friends

 

Early 2019

This is given if the MyAnimeList’s new API hasn’t been publicly released yet or people haven’t started ditching Jikan.

  • JikanPHP (Core) – Rewrite. This will introduce JikanPHP 2.X.
    • Separation of the parser as an abstraction class for Requests & RegEx parsing
    • Faster Parsing – Rework Extended Requests.
  • Jikan REST 3.0 – Given the crazy amount of requests we’ve been gettings. The main problem is rate limiting from MyAnimeList since we’re making all these requests from one server, i.e one IP Address.
    • Rework Redis Database data caching
    • API Keys. Note: This won’t replace free, unmonitored GET requests. The current limit of 5,000 will be lowered down to encourage app/project developers to get an API key that will support higher rate limits.
    • Rework Extended Requests as separate API calls. This is a bottleneck right now as extended requests make 2 requests instead of one to merge the data for you into 1 request.
  • Relational data – Expand to other sites (maybe)

Playing with Browser Extensions

So I had to test out the usability of my REST API and what’s a better way than developing an app that does it? Of course, being limited with app development  skills,  I turned to something more easier that I, atleast, have some skills in; Browser Addons/Extensions.

Now one thing I learned was that developing a  browser addons is mostly the same as making a web application. Except you need to have a manifest file and have it compiled by your browser into a proper extension. That’s pretty much it.

Enter;Anime Info

The reason why it’s such a generic name was because I had thought of the possibility of releasing it into the wild market, free to download and use. So the name would easily be search-able and would get SEO points. But the thing is it’s been about a month and Opera’s market takes almost forever to review an Addon and Chrome and Firefox have you make an initial payment to start putting addons/apps on their market, and I don’t have an card available at the moment so ¯\_(ツ)_/¯.

screen1screen2screen3

I don’t plan on updating the addon, it was merely for searching up Anime and viewing it just so I could get the gist of browser addon development and to see how my REST API worked with it. The conclusion was pretty nice.

The whole project is open-source and available on Github: https://github.com/irfan-dahir/anime-info

Designed with Material Design in mind and developed with speed in mind, it’s probably the only addon out there for its’ purpose. (No kidding, I couldn’t find anything remotely close). Sure, it can be updated to allow users to login and even update their Anime and Manga lists and even view and search Mangas – but again, nah.

Jikan API – Vision 2018 🎆 [Unofficial MyAnimeList API]

So it’s 2018 and Jikan is now 1 year old! MyAnimeList announce late 2017 that they’ll be working on fixing up their API but until then I’ll have Jikan running around. I have some plans for Jikan that need to be done, hopefully by mid-2018 or earlier, depending on college.

 

READ

 

There are some things I’m still interesting in scraping off of MAL, here’s the list.

 

User Profile

Taking an example of my own profile;

 

There’s a lot of data available per user profile. The best part here would be their favorite characters, people, anime, manga and basic stats. The hardest part to extract here would be the user based “About Me” which is highly customizable. So this, I might consider parsing since MAL’s HTML source is already terrible enough.

 

Top Anime/Manga/People/Characters

These pages give you access to a paginated list of anime/manga/people/characters ranked by their popularity/favoritism by the community from #1 to the last ranking available. Tis a gold mine entry.

 

Anime/Manga/Person/Character Search!

The official MAL API already has this feature but it only returns the first page of results! It only allows simple string queries and requires user authentication for the API call to work, which is what Jikan is meant to over come. This has been a requested feature, so I’ll most likely be working on a parser for this in the months to come.

 

 

Extended Data for Anime/Manga

This has been in the prospect of Jikan since the beginning, but I’ve held off any other extended parsing other than characters/staff and episodes until recently as I begun making scrapers for Pictures, Videos & News related to the item. This trend will continue as there are more pages that consist of interesting data regarding an anime or manga. Especially the reviews page since this has the best data for sentient analysis and averaging of any show or manga.

 

Will be focusing on these 4 for this year! It takes time to mine pure data since scraping HTML off MAL means a lot of weird and round-about ways of doing things!

Jikan REST 2.0 – Developers Preview + November Update

What a basic app utilizing Jikan would require would be data on any anime or manga, then furthermore on the characters and staff members. These 4 types of data are essential to any app for the masses and Jikan can now robustly cover any app developer in those areas.

 

Tl;Dr: https://jikan.me/api/v2.0-dev/

Note: There is no doc available for this endpoint as of yet, you’ll have to play by the data responses.

 

without further ado

It’s been a year since I started on Jikan and half a year since the REST API went up. To get this out of the way – I’m immensely excited to announce that a complete rewrite of the Jikan PHP API has been completed. Making the API more:

  • Friendly to developers for contribution
  • Cleaner responses + less bugs
  • Easier installation
  • More Robust
  • PSR-4/Autoloading

 

Now what’s left is the rewrite of the REST API. I’ve selected Lumen as the micro-framework to handle Jikan REST requests. And that’s currently in the works as I wrap my head around the features of this framework.

But my excitement could not be held back and I really wanted to see the new API in action – spitting out nicely formatted JSON without any malformed sorts of data. I quickly set up a new endpoint using the old REST API code – producing a developers endpoint.

And I hereby present: https://jikan.me/api/v2.0-dev/

You’ll notice a massive difference from the v1.1 or v1.2 REST version as this version of the API is equipped with the latest Jikan PHP commits. Now let me show you the possible type of requests.

 

  1. http://jikan.me/api/v2.0-dev/anime/1
  2. http://jikan.me/api/v2.0-dev/anime/1/characters_staff
  3. http://jikan.me/api/v2.0-dev/anime/1/episodes
  4. http://jikan.me/api/v2.0-dev/anime/21/episodes/1 – Episode pages are now paginated if there’s more than a 100 episodes, a key named episode_last_page will tell you how many pages the episodes page is paginated into.
  5. http://jikan.me/api/v2.0-dev/anime/21/episodes/2
  6. http://jikan.me/api/v2.0-dev/manga/1
  7. http://jikan.me/api/v2.0-dev/manga/1/characters
  8. http://jikan.me/api/v2.0-dev/person/1
  9. http://jikan.me/api/v2.0-dev/character/1

 

With these core prospects for the API being stable and robust, it’s time to focus on implementing more endpoints for scraping more data out of an anime, or the most required function – the search endpoint.

 

the success of this project

I’ve been contacted by a plethora number of developers regarding the usage/feedback/etc of this project. Everyone’s happy – I’m happy. There’s a working, easy to use API that can tell you anything about your favorite Japanese cartoon and I think that’s what matters the most.

Currently there’s a popular and active android App, namely AnYme that’s utilizing Jikan for their data, you can check them out here: https://github.com/zunjae/anYme

The usage of Jikan has been very successful – there’s a thousand of requests spanned across of hundreds of clients daily. Here’s a small chart on the usage since we hit off back in May.

jikan stats chart

 

what’s in store next?

The next foremost thing that is going to be accomplished is going to be REST v2.0. This will be based on the Lumen framework and a much faster server – thanks to a friend of mine. The base endpoint would be api.jikan.me, instead of what we’ve now.

After that – I’ll see what’s next on the agenda.

 

oh by the way

Did I mention that Jikan is now available on packagist.org/composer? You can install it as a dependency in your PHP project as simply as: composer require jikan-me/jikan 

Jikan – The Unofficial MyAnimeList REST API

Jikan is a REST based API that fulfills the lacking requests of the official MyAnimeList API for developers. https://jikan.me


Documentation: https://jikan.me/docs

Source: https://github.com/irfan-dahir/jikan

 

Introduction

As the idea of creating my own Anime Database sparked within me, I set out to create parse data from an existing website, MyAnimeList, since I utilize it a lot for managing the content I parse through my mind. 

Read: Making My Own Anime Database – Part 1 – Making My Own Anime Database – Part 2

I was dumbfounded when I realized that the official API did not support for fetching anime or manga details. There was a way to do this via the official API but it was totally round-about. You had to use one of their API endpoints where you searched for a query and it would return a list of similar anime/manga with their details.

I could have used AniList’s API but I was already familiar with scraping data. I’ve done this before in a lot of former projects. And so I set out to develop Jikan to fulfill my parent goal; to make my own anime database. And so it took a project of it’s own.

History

Jikan was uploaded to GitHub on January the 11th with a single function of scraping anime data.

It wasn’t even called ‘Jikan’ back then, it was called the ‘Unofficial MAL API’. Quite generic, I know.

I came to terms with the name ‘Jikan’ as it was the only domain name available for the .me TLD and it’s a commonly used word in Japanese – ‘Time’. The ‘Plan A’ name was ‘Shiro’, but unfortunately everyone seemed to have hogged all the TLDs for it.

With this API, I guess you could say I’d be saving developers some … Jikan – Heh.

 


 

Enter;Jikan

Sounds like a title from the Steins;Gate multiverse.

Anyways, Jikan can provide these details from MAL simply by their ID on MAL

  • Anime
  • Manga
  • Character
  • Person

These are the implemented functions as of now. There are some further planned features.

Search Results

The official API does support this. However;

  1. The response is in XML
  2. It only shows the results of the first page

Jikan will change that by showing results for whatever page you select. And oh – it returns in JSON.

Is that it?

Mostly, yes. The reason this API was developed to provide very easy access to developers to data which isn’t supported by the official API. And there you go.

 

Get Started

So, what are you waiting for?

Head over to the documentation and get started!

https://jikan.me