Rounding Integers Proves Troublesome

Here's the situation: I have something that is 149.95 and I want it to be represented as 14995. So, using VBScript, I take the number and multiply it by 100. On the off chance I got a number that included more than two decimal places, I wanted to lose any decimal points that might exist. VBScript has a function Int that can do just this. Every now and then, though, that whole number ends up being a number off. For example, Int(149.95 * 100) will actually give you 14994. My suspicion is that VBScript actually stores an internal number like 14994.9999999999 and when it truncates the decimal points, it leaves me with the incorrect number. I've switched to using CintCstr instead.

Published December 16, 2005 · Updated September 14, 2006
Categorized as ASP
Short URL: https://snook.ca/s/477

Conversation

16 Comments · RSS feed
Jonathan said on December 16, 2005

Have you tried using VBScript's Fix() function? I've personally never used it before, but it is supposed to strip decimals.

I've read up on the function and from what I've been seeing:

Fix(149.95 * 100)

should give you 14995.

Likewise,

Fix(149.955 * 100)

should also give you 14995.

Ben Kennedy said on December 16, 2005

Welcome to the tribulations of floating point math in a binary computer. :)

Ryan Latham said on December 16, 2005

Fix() functions almost exactlly as Int() it is another redundant Microsoft function.

However, my question here is why were you not using CInt() to begin with? Ahh the ways of a programmer, cutting corners one letter at a time.

Jonathan Snook said on December 16, 2005

Yeah, so, as I read your comment Ryan, it's been discovered that Cint creates overflow errors (likely for the same reason that int was rounding down). I've switched to Cstr which should hopefully fix it once and for all (but won't be surprised if it doesn't).

Cint(1131.95 * 100), for example, will generate an overflow error.

Ben Kennedy said on December 16, 2005

For those of us who don't speak VB, is there an online reference? I am curious to read about these various functions.

Jonathan Snook said on December 16, 2005

Ben: Check out MSDN. They have their VBScript documentation online. You can download CHM files if you like to work locally — although, you have to go through the Windows validation to download it.

Pat Allan said on December 16, 2005

Whenever I hear the word VBScript I think client-side, as server-side is categorised as ASP in my head - which is what you've put your post under.

I was extremely surprised for just a moment...

Ben Kennedy said on December 16, 2005

I thought ASP was the generic server-side framework (much like CGI) for encapsulating various scripting languages, of which VBScript is the most popular. Is that inaccurate? (I am not well versed; I stay away from Microsoft things)

Pat Allan said on December 16, 2005

I guess that would make sense - like .NET is the framework which can be coded in C++, C#, VB, Java and others.

Can't say I blame you from keeping away from Microsoft code - I've not had much luck myself, though, working with ASP and ASP.NET almost exclusively.

Jonathan Snook said on December 16, 2005

To clarify, yes, ASP is just an umbrella term and you can code in VBScript (the default) or JScript, although you can theoretically use other languages like Perl.

Ed Ferking said on December 17, 2005

See if this article helps...

http://support.microsoft.com/default.aspx?scid=kb;en-us;196652

Henrik said on December 17, 2005

Thankfully I gave up VBScript a while ago... What about the "decimal" type in C# (ASP.Net)? What about just rounding it off to closest integer? Round() that is.

Ben: ASP mostly uses VBScript, but JScript is also known, although I've never seen it. It can also use CGI, i.e. executables on the server, as far as I know (echoing jonathan), but it's different from .Net in that it doesn't first convert the language you've written to MSIL - Microsoft Intermediate Language, like the net-framework does...

Pat: Besides from VBScript (i.e. the old stuff) - what's wrong with the microsoft language c# for instance? Fully object orientated and open-standard with memory management and clean syntax.

Pat Allan said on December 17, 2005

Henrik - C# is light years ahead of VBScript, and I do love having each page as a class in ASP.NET, with code-behind to keep the XHTML clean.

That said, I've dabbled with Ruby on Rails lately, and it blows me away. It's so much quicker to develop with.

I'm also currently on site at a client's working purely with VBScript, so there's a higher pain factor at the moment.

Alexander Berglund said on September 07, 2006

I'm not sure you've seen and been trying to use round()?

<%
Round(Fix(149.95 * 100))
%>

Hmm?

Alexander Berglund said on September 07, 2006

Seemed to be an error in the post, so here goes the explaination again:

Round(Fix(149.95 * 100))

Btw; you should include a little box telling what html that is allowed and what is not.... :-)

Quinton Carroll said on March 06, 2007

How about just replacing the "." with an empty string???

Sorry, comments are closed for this post. If you have any further questions or comments, feel free to send them to me directly.