Hmm, I need to put myself back into the source code, but I may need some opinion on the data structure, or even architecture.
I'm going with a MEAN stack. MongoDB, Express and Node.js are already setup. Express will expose a RESTful API. There are 2 resources at the moments: artists and users. Authentication to the API is by API token, generated randomly per applications per users. I don't want password to remain in clear either in communication or within application ( Pixiv Fetcher for instance ). However I don't think I will step into HTTPS connections or challenge signing for auth, it's a bit overkill but who knows... Password are stored within MongoDB, hashed and salted by bcrypt.
Of course, there is an access control per user, and at the moment, it's just a power level ( mine is over 9000 :p ). 50 allows to edit artists.
Next is the model "Artist", which needs to evolve. Underscored attributes are not associated to the artist, but are necessary for the life cycle of the model
Here is what is inside MongoDB. The schema is maintained using Mongoose.
Code: Select all
{
pixivId: Number,
romanji: String,
_history: Array<Artist>,
_pixivHitNumber: Number,
_timestamp: Date,
_updater: User
}
History is merely stored as diff, and not as full state IIRC. Timestamp is the date of the last update and updater is the user who did that update.
What I consider adding:
Code: Select all
{
originalName: String or Array<String>
websites: Array<URL>,
eshuushuuTag: Number,
circles: String or Array<String>
}
What I lack for a first version:
* Full management web app, to manage users ( edit profile, increase power level, ban hammer ) and artist ( edit, browse, consult history and rollback ).
* Continue working on the Artist model data structure to totally replace the wiki. Adding an attribute with Mongoose isn't too hard
* Wiki parser so that the migration is painless ( but it's painful to create this..)
* Update Pixiv Fetcher to use API token for auth
* And while we are at it, update every library and servers
Oh and there is already a little system for create blog/news post, whatever already present.
I could also imagine publishing the source code, just need to make some adjustment to prevent passwords to get out...
Future possible problems:
First, I will certainly start to store images at some, but I need a server, and better say, where I could put R-18 loli as MoeDB is cross image board and should be extensive about its information.
Tags, I plan to translate Pixiv's tags, store E-shuushuu's tag, and other image boards as well.
Auto updating these, with robots. One robots for Pixiv.
Doujinshi, it's a pain for me to keep track of my own collection, so I'm planning to create some model for Works, in any form, doujinshi, artbook, cds, novels, whatever.
In order to maintain this, robots. I need to determine reference website. Toranoana could be one source for doujinshi for instance.
Perhaps refactor the REST API / Express server so that it automagically generates REST resources for a Mongoose schema ( but with user auth >.< )
Too many concurrent connections? Well not likely...